DEV Community: Aditya Kumar

AI Automation That Pays for Itself: What to Automate First

Aditya Kumar — Fri, 17 Jul 2026 19:37:45 +0000

The question worth answering

Not "should we use AI?" - everyone's past that. The question I get hired to answer is narrower and harder: what, specifically, is worth automating, and will it save more than it costs to build and run?

I build these systems for a living - customer support, document extraction, internal reporting, ticket routing, in Laravel, Node and Python. The pattern that decides whether one of them earns its keep has nothing to do with the model or the framework. It is picking the right work, and engineering it so it survives contact with real users. Sometimes the most valuable line in the report is: don't automate this one.

I'll walk through how I decide what to automate, four systems worth building (with the engineering that matters and the numbers that matter), what it really costs, and the failure mode almost nobody budgets for.

The band worth automating

The work worth automating sits in a narrow band: high-volume, repetitive, and judgment-light. Your team does it dozens of times a day, the steps rarely change, and a wrong answer is annoying but recoverable.

Good fit: "Where's my order?" replies. Extracting three fields from every invoice. First-pass ticket triage. Turning a call transcript into CRM fields. The standard follow-up email.
Bad fit: Anything rare - you'll spend more building than you'll ever save. Anything where a wrong answer costs money or trust: pricing, contracts, legal or medical calls, irreversible actions. Anything a person does because they're good at it, not because it's tedious.

Founders get this backwards constantly. They want to automate the impressive, judgment-heavy work - "an AI that handles our hardest customers." That's the expensive, risky end, and it's where AI is weakest. The return is in the boring middle: the hundred small, identical tasks that quietly eat a person's afternoon. As an engineer, I want the boring problems - they're the ones I can make reliable.

Four systems worth building

1. Support triage that looks things up instead of guessing

Most inbound support to a small e-commerce or SaaS team is the same handful of questions: order status, "how do I reset X," "do you support Y." A person reads each one, looks something up, and types a reply they've typed a hundred times.

The automation is not "a chatbot that talks to customers." It's narrower and far more useful: an assistant that reads the message, looks up the real answer in your own systems, and drafts a reply for a human to approve or send. It handles the repetitive majority; the person spends their time on the minority that genuinely needs a human.

The engineering that makes it safe: the model never invents an order number, because the number comes from a tool call into your database, scoped in code to the authenticated customer - not from the model's imagination. The model decides which tool it needs; your code decides what it's allowed to see. That authorization boundary is the whole game, and it lives in your code, never in the prompt. (I wrote up the tool-calling loop in detail - it's a few hundred lines of plain application code, whichever stack you're on. No platform to buy.)

What it costs: well under a cent per resolved conversation on a mid-tier model, in my production experience. At a thousand messages a month the model bill is single-digit dollars; the person-hours handed back are the actual return and they dwarf it. The gotcha: keep a human approving the drafts until you've watched a few hundred and trust the quality. Ship it as a draft-assistant first, an auto-responder second - if ever.

2. Document extraction with a confidence threshold

Invoices, receipts, purchase orders, resumes - structured data trapped in unstructured PDFs. Someone opens each file and retypes three to ten fields into a system. It's soul-destroying and error-prone, and it's a near-perfect automation target.

The engineering: instead of asking the model for prose, you demand structured output against a JSON schema - invoice_number, total, due_date, line_items - and the model returns data your code can validate, not a paragraph you have to parse. Then the critical piece founders skip: a confidence threshold. The model returns its own uncertainty (or you derive it from validation - does the total match the line items?). High-confidence extractions flow straight through; low-confidence ones get flagged for a human. You're not replacing the person; you're shrinking their queue from every document to the hard 10%.

What it costs: a fraction of a cent per document; the build is a few days once the schema is nailed down. The gotcha: the hard part isn't the AI, it's your data. If your "invoices" arrive as scanned photos at three resolutions, half the work is image handling before the model ever sees them. I quote that honestly up front, because it's where these projects actually slow down.

3. "Ask your data" internal reporting

Every founder wants to type "how many orders shipped from the Delhi warehouse last week?" and get an answer, instead of pinging the one engineer who can write the query. Done naively this is dangerous. Done right it's one of the highest-ROI internal tools you can ship.

The engineering: the model translates the question into a tool call against a read-only, permission-scoped set of queries you defined - not free-form SQL it wrote itself (never let a model write arbitrary SQL against production; that's a breach with extra steps). You expose a handful of safe, parameterized queries as tools; the model picks and fills them; your code runs them under the asker's permissions. Cache the common ones. Return numbers, not prose the model might round wrong.

What it costs: pennies per query, and it deletes a whole category of "quick question for engineering" interruptions. The gotcha: scope every query to the asker's role. An "ask your data" tool that ignores permissions is a data-leak generator with a friendly chat UI.

4. Ticket classification and routing

Before a human ever reads a ticket, something has to decide: is this billing, bug, sales, or spam - and how urgent? Teams do this by hand, badly, all day.

The engineering: this one's often cheaper and better without the heaviest model. A small, fast model (or even a fine-tuned classifier) tags category + priority in a few hundred milliseconds for a fraction of a cent. The win isn't magic - it's consistency and speed at volume. But it only stays good if you measure it: a labelled eval set of real tickets you re-run whenever you change the prompt or swap the model, so "I tweaked the wording" can't silently drop accuracy 12%.

What it costs: the cheapest of the four to run; the real investment is building and maintaining the eval set. The gotcha: without evals you're flying blind, and classification drift is invisible until a VIP's urgent ticket sits in the wrong queue for two days.

What AI automation actually costs

Founders' intuition is usually wrong in both directions here. They overestimate the API bill and underestimate the maintenance.

Running cost is genuinely small. The systems above run from a fraction of a cent to a couple of cents per action on mid-tier models. Unless you're at serious scale, the model bill is a rounding error next to the salaries it frees up. The honest framing: it costs more per minute to have a person retype the invoice than to have the model do it and a person spot-check it.

Latency is the cost founders forget. A tool-calling agent makes multiple round-trips to the model - each one carrying the whole conversation plus your tool definitions - so a three-step answer can take five to fifteen seconds. Fine for a drafted email; wrong for something a customer stares at live. Part of my job is knowing when to stream partial output, when to cap the loop (I hard-limit agents to five rounds - deeper is a design smell, not a feature), and when the answer is "run this in a background job, not in the request."

Build cost is a few days to a few weeks, driven almost entirely by how many systems it touches and how clean your data is. The clean-data part is where timelines actually go - if your order data lives in three places that disagree, that's the real project, and AI doesn't fix it.

Maintenance is the line item nobody budgets for. Models change. Your product changes. The system that was right in January drifts by June. You need logging on every AI decision, a spend cap so a runaway loop can't bill you $500 overnight, and evals so you notice quality slipping before your customers do. Budget for watching it, not just building it. An unwatched automation is a liability with a monthly invoice.

The failure mode nobody warns you about

It's not hallucination - the tool-grounding above mostly handles that. It's silent partial failure.

It works at launch. That's the trap. Six weeks later the model is quietly getting some fraction of answers wrong - call it 10 or 15% - not wrong enough for anyone to complain loudly, just wrong enough to erode trust one interaction at a time. Nobody notices, because everyone's watching uptime, not outputs.

The fix is unglamorous and non-negotiable, and it's exactly what separates the automations that last from the ones that quietly rot: log every decision (input, action taken, output), sample them on a schedule, keep a labelled eval set you re-run on every change, and hold a human approval step in front of anything customer-facing until you've earned the confidence to remove it. The systems that survive are the ones you can audit. The ones that fail are the ones someone set and forgot.

The engineer's checklist (steal this before you greenlight anything)

Before you approve an AI automation, get honest answers to these - I run every client engagement through them:

Volume: Does someone do this 20+ times a day? If it's rare, skip it.
Judgment: Is the "right answer" mostly lookup-and-format, or does it need real human judgment? Automate the former.
Cost of wrong: At a 5% error rate, what does a mistake cost? If the answer is "a lot," keep a human in the loop or don't build it.
Grounding: Can the answer come from a tool call into your real data, instead of the model's memory? If not, expect it to make things up.
Data: Does that data live somewhere you can query cleanly? If it's a mess, that's project one, not the AI.
Watchability: Can you log, sample, and eval every decision? If not, you're flying blind.

Clear all six and it's a strong candidate. Fail two or more and the honest answer is often "a queue and a cron job" - plain automation, no model - and I'll tell a client that rather than sell them AI they don't need.

Where to start

Pick one workflow: the most boring, highest-volume, lowest-stakes thing your team does. Automate that, keep a human approving the output, watch it for a month, and measure the hours it hands back. That single loop will teach you more about whether AI fits your business than any pilot deck.

That's what my AI Integration Audit is: one week, fixed price, I look at your real workflows and tell you which two or three are worth automating, what each will cost to build and run, and which ones to leave alone. Sometimes the most valuable line in that report is "don't automate this one." Founders remember the engineer who told them that.

I'm Aditya Kumar (adityakdevin) - Tech Lead & full-stack developer building AI-powered web products with Laravel, Vue, and LLM APIs. Find me at adityadev.in.

Building AI Agents in PHP: Tool Calling with Laravel

Aditya Kumar — Fri, 17 Jul 2026 19:37:25 +0000

The problem: chat that can only talk

The chatbot from earlier in this series can answer questions, but ask it "how many orders did we ship yesterday?" and it will confidently make something up. It has no hands — it can't query your database, call your services, or do anything except generate text.

Tool calling fixes that. You describe functions to the model; when a user's request needs one, the model responds with "call this function with these arguments" instead of prose. Your code runs the function, feeds the result back, and the model writes the final answer grounded in real data. That loop — model picks tool, you execute, model continues — is the whole trick behind "agents." No framework required.

I've shipped this in production Laravel apps for order lookups, report generation, and support triage. Here's the pattern that survived contact with real users.

Architecture

Three pieces, all plain Laravel:

User message ──→ AgentController
                     │
                     ▼
              AgentRunner (the loop, max N rounds)
                     │
        ┌── model returns text? ──→ done, return answer
        │
        └── model returns tool_calls?
                     │
                     ▼
              ToolRegistry ──→ OrderLookupTool
                            ──→ ShippingStatusTool
                     │              (each: name, schema, handle())
                     ▼
              results appended to messages ──→ back to the model

Each tool is one class. The registry maps names to classes. The runner owns the loop and the guardrails. That's it.

Step 1: a tool contract

// app/Ai/Tools/Tool.php
interface Tool
{
    public function name(): string;

    public function description(): string;

    /** JSON Schema for the arguments the model may pass. */
    public function parameters(): array;

    /** @param array $args validated arguments from the model */
    public function handle(array $args): string;
}

Tools return strings because that's what goes back into the conversation. JSON-encode structured data — models read it fine.

Step 2: a real tool

// app/Ai/Tools/OrderLookupTool.php
class OrderLookupTool implements Tool
{
    public function name(): string
    {
        return 'order_lookup';
    }

    public function description(): string
    {
        return 'Look up an order by its number. Returns status, items, and shipping info.';
    }

    public function parameters(): array
    {
        return [
            'type' => 'object',
            'properties' => [
                'order_number' => [
                    'type' => 'string',
                    'description' => 'The order number, e.g. ORD-2041',
                ],
            ],
            'required' => ['order_number'],
        ];
    }

    public function handle(array $args): string
    {
        $order = Order::where('number', $args['order_number'])
            ->where('user_id', auth()->id())   // ← the line that matters
            ->first();

        if (! $order) {
            return json_encode(['error' => 'Order not found for this account.']);
        }

        return json_encode([
            'number' => $order->number,
            'status' => $order->status,
            'items' => $order->items->pluck('name'),
            'shipped_at' => $order->shipped_at?->toDateString(),
        ]);
    }
}

That auth()->id() scope is the single most important line in this post. The model chooses which tool to call and what arguments to pass — an attacker can steer both with a crafted prompt. Authorization must live in your code, never in the prompt. Treat every tool as a public API endpoint, because that's what it is now.

Step 3: the loop

// app/Ai/AgentRunner.php
class AgentRunner
{
    private const MAX_ROUNDS = 5;

    public function __construct(private ToolRegistry $tools) {}

    public function run(array $messages): string
    {
        foreach (range(1, self::MAX_ROUNDS) as $round) {
            $response = Http::withToken(config('services.openai.key'))
                ->timeout(30)
                ->post('https://api.openai.com/v1/chat/completions', [
                    'model' => 'gpt-4.1-mini',
                    'messages' => $messages,
                    'tools' => $this->tools->schemas(),
                ])->throw()->json('choices.0.message');

            // Plain text answer — we're done.
            if (empty($response['tool_calls'])) {
                return $response['content'] ?? '';
            }

            $messages[] = $response;

            foreach ($response['tool_calls'] as $call) {
                $result = $this->tools->execute(
                    $call['function']['name'],
                    json_decode($call['function']['arguments'], true) ?? [],
                );

                $messages[] = [
                    'role' => 'tool',
                    'tool_call_id' => $call['id'],
                    'content' => $result,
                ];
            }
        }

        return 'I could not complete that in a reasonable number of steps.';
    }
}

MAX_ROUNDS is not optional. Without it, a confused model can ping-pong between tools forever — each round costing you tokens and your user thirty seconds of spinner. Five rounds covers every legitimate flow I've shipped; anything deeper is a design smell.

The registry's execute() is where you validate:

public function execute(string $name, array $args): string
{
    $tool = $this->map[$name] ?? null;

    if (! $tool) {
        return json_encode(['error' => "Unknown tool: {$name}"]);
    }

    try {
        return $tool->handle($args);
    } catch (Throwable $e) {
        report($e);

        return json_encode(['error' => 'Tool failed. Try rephrasing.']);
    }
}

Return errors to the model as tool results instead of throwing. Models handle "that didn't work" gracefully — they retry with fixed arguments or tell the user. An unhandled exception, by contrast, kills the whole conversation.

What it costs

Every round is a full API call carrying the entire conversation plus your tool schemas. A three-round agent turn with five registered tools runs 3–5× the tokens of a plain chat reply. Two mitigations that pay for themselves immediately: register only the tools relevant to the current context (a support agent doesn't need admin reporting tools), and keep descriptions tight — the model reads every schema on every round. On gpt-4.1-mini, my production order-support agent averages under a cent per resolved conversation. The engineer time it replaced cost more per minute.

Production checklist

Authorization inside every tool. Scope queries to the authenticated user. The model is an untrusted caller with a keyboard.
Cap the loop. MAX_ROUNDS, request timeouts, and a per-user rate limit on the endpoint.
Validate arguments. json_decode the model's arguments defensively; missing keys and wrong types are routine, not exceptional.
Log every tool call. Name, arguments, duration, result size. When the agent does something weird — and it will — this log is how you find out what and why.
Return errors as data. Tool failures become tool results; the model recovers. Exceptions become 500s; the user leaves.
Test tools like controllers. Each handle() is plain PHP — unit test it directly, then one integration test with Http::fake() for the loop itself.

Where this goes next

The agent above runs synchronously — fine for lookups, wrong for anything slow. When a tool takes thirty seconds or the user asks for a report across a million rows, you need the loop running in a queued job with progress updates, retries, and a budget. That's the next post in this series: Queue-Based AI Workflows in Laravel — Jobs, Retries, and Cost Control.

I'm Aditya Kumar (adityakdevin) — Tech Lead & full-stack developer building AI-powered web products with Laravel, Vue, and LLM APIs. Find me at adityadev.in.

RAG in Laravel: Embeddings and pgvector for a Knowledge-Base Bot

Aditya Kumar — Thu, 16 Jul 2026 21:09:06 +0000

In the last post we streamed AI responses over SSE. Now let's fix the problem every chatbot hits in week one: the model doesn't know your data. Ask it about your refund policy or your API docs and it either hallucinates or shrugs.

RAG — retrieval-augmented generation — fixes that. Instead of hoping the model knows your content, you store your documents as embeddings, retrieve the few chunks relevant to each question, and hand only those to the model. Here's the whole pipeline in Laravel 11 with PostgreSQL and pgvector — no vector-database SaaS required.

First, an honest gate: do you even need RAG?

I recently shipped a chatbot for my own portfolio site. Total knowledge base: about 2,000 tokens of facts and FAQ. I didn't build RAG for it — I put the entire corpus in the system prompt and let prompt caching make it cheap. Simpler code, zero retrieval bugs, and the model sees everything on every question.

The rule of thumb I use with clients:

Corpus under ~20k tokens → whole corpus in a cached system prompt. Done.
Corpus larger than that, or updated constantly → RAG.

If you're in the first bucket, close this tab and go ship. Still here? Let's build.

The stack

Laravel 11, PostgreSQL 15+ with the pgvector extension
openai-php/laravel for embeddings + chat
pgvector/pgvector-php for the Eloquent cast

composer require openai-php/laravel pgvector/pgvector
php artisan vendor:publish --provider="OpenAI\Laravel\ServiceProvider"

On most managed Postgres (RDS, Supabase, Neon, Laravel Forge boxes) pgvector is one statement away:

CREATE EXTENSION IF NOT EXISTS vector;

The schema

One table stores chunks of your documents plus their embedding vectors. text-embedding-3-small produces 1536 dimensions — good quality, cheap ($0.02 per million tokens).

use Illuminate\Database\Migrations\Migration;
use Illuminate\Support\Facades\DB;
use Illuminate\Support\Facades\Schema;

return new class extends Migration
{
    public function up(): void
    {
        DB::statement('CREATE EXTENSION IF NOT EXISTS vector');

        Schema::create('document_chunks', function ($table) {
            $table->id();
            $table->string('source');          // e.g. "refund-policy.md"
            $table->unsignedInteger('position'); // chunk order within the source
            $table->text('content');
            $table->timestamps();
        });

        DB::statement('ALTER TABLE document_chunks ADD COLUMN embedding vector(1536)');

        // HNSW index makes similarity search fast past ~10k rows
        DB::statement('CREATE INDEX ON document_chunks USING hnsw (embedding vector_cosine_ops)');
    }
};

The model:

namespace App\Models;

use Illuminate\Database\Eloquent\Model;
use Pgvector\Laravel\Vector;

class DocumentChunk extends Model
{
    protected $fillable = ['source', 'position', 'content', 'embedding'];

    protected $casts = ['embedding' => Vector::class];
}

Ingestion: chunk, embed, store

Chunking strategy matters more than people admit. My default: split on headings/paragraphs, target ~500 tokens per chunk, and never split mid-sentence. Fancy overlap windows can wait until you have evidence you need them.

namespace App\Console\Commands;

use App\Models\DocumentChunk;
use Illuminate\Console\Command;
use Illuminate\Support\Facades\File;
use OpenAI\Laravel\Facades\OpenAI;

class IngestDocs extends Command
{
    protected $signature = 'rag:ingest {path : Directory of markdown files}';

    public function handle(): int
    {
        foreach (File::files($this->argument('path')) as $file) {
            $chunks = $this->chunk($file->getContents());

            // One API call per file, not per chunk — batch input is supported
            $response = OpenAI::embeddings()->create([
                'model' => 'text-embedding-3-small',
                'input' => $chunks,
            ]);

            DocumentChunk::where('source', $file->getFilename())->delete();

            foreach ($response->embeddings as $i => $embedding) {
                DocumentChunk::create([
                    'source' => $file->getFilename(),
                    'position' => $i,
                    'content' => $chunks[$i],
                    'embedding' => $embedding->embedding,
                ]);
            }

            $this->info("{$file->getFilename()}: " . count($chunks) . ' chunks');
        }

        return self::SUCCESS;
    }

    /** @return string[] */
    private function chunk(string $text, int $targetChars = 2000): array
    {
        $paragraphs = preg_split('/\n{2,}/', $text);
        $chunks = [''];

        foreach ($paragraphs as $p) {
            $current = array_key_last($chunks);
            if (strlen($chunks[$current]) + strlen($p) > $targetChars && $chunks[$current] !== '') {
                $chunks[] = $p;
            } else {
                $chunks[$current] .= "\n\n" . $p;
            }
        }

        return array_map('trim', $chunks);
    }
}

Run it: php artisan rag:ingest storage/docs. Re-running replaces a file's chunks, so ingestion is idempotent — wire it to your deploy or a scheduled job and content stays fresh.

Retrieval: the five-line heart of RAG

pgvector's <=> operator is cosine distance. Lower = more similar. Embed the user's question, sort by distance, take the top handful:

namespace App\Services;

use App\Models\DocumentChunk;
use OpenAI\Laravel\Facades\OpenAI;

class Retriever
{
    /** @return array{content: string, source: string}[] */
    public function search(string $question, int $limit = 5): array
    {
        $embedding = OpenAI::embeddings()->create([
            'model' => 'text-embedding-3-small',
            'input' => $question,
        ])->embeddings[0]->embedding;

        return DocumentChunk::query()
            ->selectRaw('content, source, embedding <=> ? AS distance', [json_encode($embedding)])
            ->orderBy('distance')
            ->limit($limit)
            ->get()
            ->filter(fn ($c) => $c->distance < 0.55) // relevance floor — tune on real queries
            ->map(fn ($c) => ['content' => $c->content, 'source' => $c->source])
            ->all();
    }
}

That distance floor is your hallucination guard. If nothing scores under the threshold, the honest answer is "I don't know" — and you should say exactly that instead of feeding the model junk context.

Answering with context

namespace App\Services;

use OpenAI\Laravel\Facades\OpenAI;

class KnowledgeBot
{
    public function __construct(private Retriever $retriever) {}

    public function answer(string $question): string
    {
        $chunks = $this->retriever->search($question);

        if ($chunks === []) {
            return "I couldn't find that in our documentation — try rephrasing, or contact support.";
        }

        $context = collect($chunks)
            ->map(fn ($c) => "[{$c['source']}]\n{$c['content']}")
            ->implode("\n\n---\n\n");

        $response = OpenAI::chat()->create([
            'model' => 'gpt-4o-mini',
            'max_tokens' => 500,
            'messages' => [
                ['role' => 'system', 'content' => <<<PROMPT
                    Answer using ONLY the context below. If the context doesn't
                    contain the answer, say you don't know — never invent facts.
                    Cite the source file name when you answer.

                    Context:
                    {$context}
                    PROMPT],
                ['role' => 'user', 'content' => $question],
            ],
        ]);

        return $response->choices[0]->message->content;
    }
}

Wire KnowledgeBot::answer() into the streaming controller from part 2 of this series and you have a knowledge-base bot that streams grounded answers.

Production checklist

Relevance floor: filter retrieved chunks by distance threshold; return an honest "not found" below it. Tune the number against real user queries, not vibes.
Idempotent ingestion: delete-and-replace per source file, run on deploy. Stale embeddings answer yesterday's questions.
HNSW index: without it, every query is a full table scan. With it, sub-10ms at hundreds of thousands of rows.
Batch embeddings: one API call per document, not per chunk — 10x fewer requests and the same price.
Cost caps and rate limits: everything from part 1 still applies — each question costs one embedding call plus one chat call.
Log question + retrieved sources (not full content): when an answer is wrong, you need to know whether retrieval or generation failed. That distinction decides your fix.
Evaluate retrieval separately: keep 20 known question→source pairs and assert the right chunk lands in the top 5. This catches chunking regressions before your users do.

What's next

Retrieval makes the model knowledgeable. The next step makes it capable: in part 4 we'll build AI agents in PHP — giving the model tools it can call (search orders, create tickets, query your database) and handling the tool-calling loop safely in Laravel. That's where chatbots turn into products.

I'm Aditya Kumar (adityakdevin) — Tech Lead & full-stack developer building AI-powered web products with Laravel, Vue, and LLM APIs. Find me at adityadev.in.

Streaming AI Responses in Laravel with Server-Sent Events

Aditya Kumar — Thu, 16 Jul 2026 07:49:29 +0000

In the first post of this series we built a working AI chatbot in Laravel. It had one problem every user notices immediately: you send a message, then stare at a spinner for five seconds while the whole response generates.

LLMs produce text token by token. If you wait for the full completion before showing anything, you're throwing away the single biggest UX win available: streaming. The same answer that takes 5 seconds to finish starts appearing in ~300ms when streamed. Nothing about the model got faster — but to the user, it feels 10x faster.

In this post we'll upgrade the chatbot to stream responses with Server-Sent Events (SSE) — no WebSockets, no Pusher, no extra infrastructure. Just Laravel.

Why SSE and not WebSockets?

WebSockets are bidirectional and need a long-running server (Reverb, Soketi) or a paid service. For chat completions you only need one direction: server → browser, for the lifetime of one request. That's exactly what SSE is for, and since Laravel 11 it's built into the framework as response()->eventStream().

Rule of thumb: presence, typing indicators, multiplayer → WebSockets. Streaming one AI answer → SSE.

1. Stream from the OpenAI client

We keep the ChatService from part 1 and add a streaming method. The openai-php client exposes createStreamed(), which returns an iterator of deltas:

<?php

namespace App\Services;

use Generator;
use OpenAI\Laravel\Facades\OpenAI;

class ChatService
{
    // ... SYSTEM_PROMPT and reply() from part 1 ...

    /**
     * @param array<int, array{role: string, content: string}> $history
     */
    public function streamReply(array $history, string $userMessage): Generator
    {
        $stream = OpenAI::chat()->createStreamed([
            'model' => 'gpt-4o-mini',
            'messages' => [
                ['role' => 'system', 'content' => self::SYSTEM_PROMPT],
                ...$history,
                ['role' => 'user', 'content' => $userMessage],
            ],
            'max_tokens' => 500,
        ]);

        foreach ($stream as $response) {
            $delta = $response->choices[0]->delta->content;

            if ($delta !== null) {
                yield $delta;
            }
        }
    }
}

A Generator is the right return type here: the controller can forward chunks as they arrive without ever holding the full response in memory.

2. The streaming controller

Laravel 11.19+ ships response()->eventStream(), which handles the SSE formatting (event: / data: lines) for you:

<?php

namespace App\Http\Controllers;

use App\Services\ChatService;
use Illuminate\Http\Request;

class ChatStreamController extends Controller
{
    public function __invoke(Request $request, ChatService $chat)
    {
        $validated = $request->validate([
            'message' => ['required', 'string', 'max:2000'],
        ]);

        $history = $request->session()->get('chat_history', []);

        // CRITICAL: release the session lock. A streaming response is a
        // long-lived request — with the default file/database session
        // driver it would block every other request from this user
        // (including page loads!) until the stream finishes.
        $request->session()->save();

        return response()->eventStream(function () use ($request, $chat, $history, $validated) {
            $full = '';

            foreach ($chat->streamReply($history, $validated['message']) as $chunk) {
                $full .= $chunk;
                yield $chunk;
            }

            // persist history once the stream completes
            $history[] = ['role' => 'user', 'content' => $validated['message']];
            $history[] = ['role' => 'assistant', 'content' => $full];
            $request->session()->put('chat_history', array_slice($history, -20));
            $request->session()->save();
        }, headers: [
            'X-Accel-Buffering' => 'no', // tell nginx not to buffer the stream
        ]);
    }
}

Route it with the same throttling as part 1:

Route::post('/chat/stream', ChatStreamController::class)
    ->middleware(['auth', 'throttle:20,1']);

3. Reading the stream in the browser

The native EventSource API only supports GET requests, and we need to POST a message body — so we read the stream with fetch instead:

<script>
async function streamChat(message, botEl) {
    const res = await fetch('/chat/stream', {
        method: 'POST',
        headers: {
            'Content-Type': 'application/json',
            'X-CSRF-TOKEN': document.querySelector('meta[name="csrf-token"]').content,
        },
        body: JSON.stringify({ message }),
    });

    const reader = res.body.getReader();
    const decoder = new TextDecoder();
    let buffer = '';

    while (true) {
        const { done, value } = await reader.read();
        if (done) break;

        buffer += decoder.decode(value, { stream: true });

        // SSE messages are separated by a blank line
        const events = buffer.split('\n\n');
        buffer = events.pop(); // keep the incomplete tail

        for (const evt of events) {
            const data = evt.split('\n')
                .filter(l => l.startsWith('data: '))
                .map(l => l.slice(6))
                .join('\n');

            if (data && data !== '</stream>') {
                botEl.textContent += data;
            }
        }
    }
}
</script>

</stream> is Laravel's default end-of-stream marker — filter it out (or customize it with the endStreamWith argument).

That's it. Send a message and watch the answer type itself out.

Where streaming breaks in production

This is the part most tutorials skip. Streaming works instantly on php artisan serve, then mysteriously arrives all-at-once on your server. The culprit is always buffering somewhere between PHP and the browser:

nginx buffers proxied responses by default. The X-Accel-Buffering: no header above disables it per-response; alternatively set proxy_buffering off; for the route.
PHP-FPM + output buffering: check output_buffering in php.ini. Laravel's eventStream flushes after every yield, but an outer buffer can still swallow it.
Cloudflare / load balancers: most respect SSE content types, but verify with curl -N against production before blaming your code.

Debug tip: curl -N -X POST https://yourapp.test/chat/stream ... shows you exactly what arrives and when, with no browser magic in between.

Production checklist

Release the session lock before streaming (done above) — this is the #1 "my app hangs" bug with SSE in Laravel.
X-Accel-Buffering: no header for nginx (done above).
Timeouts: a stream can outlive max_execution_time and your web server's send timeout. Set both above your worst-case generation time (60s is a sane ceiling with max_tokens: 500).
Mid-stream errors: wrap the generator loop in try/catch and yield a friendly "Something went wrong" chunk — a dead stream with no message looks frozen to users.
Rate limit the endpoint (done above) — streaming doesn't change the fact that every request costs money.
Don't stream everything: for background jobs or short answers, the buffered endpoint from part 1 is simpler. Keep both.

Next in this series

Our bot streams beautifully, but it still only knows what's in its system prompt. Next up: RAG in Laravel — embeddings and pgvector, where we'll give it your actual documentation to answer from. Follow me to catch it.

I'm Aditya Kumar (adityakdevin) — Tech Lead & full-stack developer building AI-powered web products with Laravel, Vue, and LLM APIs. Find me at adityadev.in.

Adding an AI Chatbot to Your Laravel App with the OpenAI API

Aditya Kumar — Thu, 16 Jul 2026 07:32:50 +0000

Every product I've worked on in the last two years has eventually gotten the same request: "Can we add an AI assistant to this?" As a Laravel developer, the good news is that shipping a production-ready chatbot takes an afternoon — no Python microservice, no LangChain, just Laravel and an HTTP API.

In this tutorial we'll build a chatbot that:

answers questions in the context of your product (via a system prompt),
remembers the conversation within a session,
streams nothing fancy — just clean JSON in/out you can drop into any Blade view or SPA.

1. Install the OpenAI PHP client

The community-maintained openai-php/laravel package is the de-facto standard:

composer require openai-php/laravel
php artisan openai:install

Add your key to .env:

OPENAI_API_KEY=sk-your-key-here

💡 Never commit the key. Use your host's secret manager in production.

2. A dedicated chat service

Keep the LLM logic out of your controller. Create app/Services/ChatService.php:

<?php

namespace App\Services;

use OpenAI\Laravel\Facades\OpenAI;

class ChatService
{
    private const SYSTEM_PROMPT = <<<'PROMPT'
        You are the helpful assistant for Acme Inc's customer portal.
        Answer only questions about the product. Be concise.
        If you don't know the answer, say so and suggest contacting support.
        PROMPT;

    /**
     * @param array<int, array{role: string, content: string}> $history
     */
    public function reply(array $history, string $userMessage): string
    {
        $messages = [
            ['role' => 'system', 'content' => self::SYSTEM_PROMPT],
            ...$history,
            ['role' => 'user', 'content' => $userMessage],
        ];

        $response = OpenAI::chat()->create([
            'model' => 'gpt-4o-mini',
            'messages' => $messages,
            'max_tokens' => 500,
        ]);

        return $response->choices[0]->message->content;
    }
}

3. The controller

Session-based history keeps things simple — no migrations needed for v1:

<?php

namespace App\Http\Controllers;

use App\Services\ChatService;
use Illuminate\Http\JsonResponse;
use Illuminate\Http\Request;

class ChatController extends Controller
{
    public function __invoke(Request $request, ChatService $chat): JsonResponse
    {
        $validated = $request->validate([
            'message' => ['required', 'string', 'max:2000'],
        ]);

        $history = $request->session()->get('chat_history', []);

        $answer = $chat->reply($history, $validated['message']);

        // keep the last 10 exchanges to control token costs
        $history[] = ['role' => 'user', 'content' => $validated['message']];
        $history[] = ['role' => 'assistant', 'content' => $answer];
        $request->session()->put('chat_history', array_slice($history, -20));

        return response()->json(['answer' => $answer]);
    }
}

Route it — with rate limiting, because every request costs you money:

// routes/web.php
use App\Http\Controllers\ChatController;

Route::post('/chat', ChatController::class)
    ->middleware(['auth', 'throttle:20,1']); // 20 req/min per user

4. A minimal frontend

Drop this into any Blade view — no framework required:

<div id="chat">
    <div id="messages"></div>
    <form id="chat-form">
        <input id="chat-input" type="text" placeholder="Ask me anything…" required />
        <button type="submit">Send</button>
    </form>
</div>

<script>
document.getElementById('chat-form').addEventListener('submit', async (e) => {
    e.preventDefault();
    const input = document.getElementById('chat-input');
    const messages = document.getElementById('messages');

    messages.insertAdjacentHTML('beforeend', `<p><b>You:</b> ${input.value}</p>`);

    const res = await fetch('/chat', {
        method: 'POST',
        headers: {
            'Content-Type': 'application/json',
            'X-CSRF-TOKEN': document.querySelector('meta[name="csrf-token"]').content,
        },
        body: JSON.stringify({ message: input.value }),
    });

    const { answer } = await res.json();
    messages.insertAdjacentHTML('beforeend', `<p><b>Bot:</b> ${answer}</p>`);
    input.value = '';
});
</script>

5. Production checklist

Before you ship this to real users:

Rate limit (done above) — an unthrottled chat endpoint is an open wallet.
Cap history (done above) — token usage grows with every message otherwise.
Validate input length (done above) — 2000 chars is plenty for a question.
Log conversations — a simple chat_logs table pays for itself the first time you debug a weird answer.
Set max_tokens — bounds your worst-case cost per request.
Add a fallback — wrap the API call in a try/catch and return "I'm having trouble right now" instead of a 500.

Where to go next

Streaming responses — OpenAI::chat()->createStreamed() + Server-Sent Events makes the bot feel 10x faster.
RAG (Retrieval-Augmented Generation) — embed your docs and inject relevant chunks into the system prompt so the bot answers from your knowledge base. That's my next post.
Swap providers — the same pattern works with Anthropic's Claude API; only the HTTP client changes.

I'm Aditya Kumar (adityakdevin) — Tech Lead & full-stack developer building AI-powered web products with Laravel, Vue, and LLM APIs. Find me at adityadev.in.