🚀 Technical Briefing: This tutorial is part of our deep-dive series on Agentic Workflows at Gate of AI. For the full technical breakdown, interactive code sandbox, and the native Arabic translation, visit the original article here.
Technical tutorial · RAG ticket resolution · Intermediate
Build a Safer Next.js GPT-4 RAG Ticket Router
Design a support-ticket routing workflow that retrieves relevant historical engineering evidence before asking a GPT-4-class model for a recommendation. This version focuses on the capabilities and risks verified in the supplied research: semantic retrieval, re-ranking, evidence-grounded generation, and protection against false claims that an action was completed.
What This Tutorial Actually Builds
The goal is not an autonomous support bot that silently changes queues or submits tickets. The goal is a reviewable recommendation pipeline for a Next.js application. A support message enters the system, related tickets and engineering artifacts are retrieved, the evidence is re-ranked, and a GPT-4-class generation layer proposes a classification or resolution path. A human or a separately verified application workflow must confirm any operational action.
This distinction matters. The verified research on integrated task and knowledge assistants reports that GPT-4 function-calling systems sometimes respond as though an action was completed when the submission function was not used. It also reports premature API invocation, failure to collect all required fields, and hallucinated responses in ticket-submission scenarios. Therefore, a model response must be treated as a recommendation until the application verifies a real tool result.
The design is also different from a prompt-only classifier. RAG4Tickets describes a pipeline in which a ticket is encoded as an embedding, semantically similar tickets, pull requests, and user comments are retrieved, the retrieved artifacts are re-ranked using cosine similarity and contextual overlap, and the selected material is passed to the generation layer. That is the architecture used here.
Prerequisites and Evidence Boundaries
You should already understand the basic structure of a Next.js application, server-side request handling, vector search, and the difference between a model response and a confirmed application action. The supplied context does not verify a particular Next.js release, OpenAI JavaScript package version, hosting provider, or database. Do not copy version claims from an older tutorial into production without checking the current official documentation for your chosen stack.
The verified research names GPT-4, Claude, and LLaMA 3 as examples of generation models. It does not verify GPT-4o specifically, a particular API endpoint, or a particular structured-output feature. For that reason, this tutorial describes the model boundary generically as an approved GPT-4-class generation service. Before implementation, your engineering team must confirm the exact provider API, authentication method, response contract, and data-processing terms.
The same evidence boundary applies to performance. The reported experiment used approximately 1.2 million tickets and pull requests and found that HNSW reduced average query latency by 73% compared with a Flat index, with a 2% reduction in recall. That is useful research context, not a promise for your dataset, infrastructure, language mix, or workload.
Step 1: Define the Routing Decision Before Calling a Model
Begin with the finite decision your support organization needs. Examples include an issue category, candidate queue, urgency recommendation, evidence references, and a review flag. Do not begin by asking the model to “solve the ticket.” A broad objective makes it difficult to evaluate whether the system classified the issue, found relevant evidence, proposed a resolution, or merely produced persuasive prose.
Write the allowed values in a product or engineering specification. For example, a team might define categories for billing, access, security, technical failure, and general support. Those labels are illustrative and must be replaced with the taxonomy used by your organization. The important constraint is that the model must select from an approved vocabulary rather than inventing a destination.
Define required fields as well. A ticket-submission workflow may require a title, description, customer identifier, impact, environment, and confirmation status. The research warns that GPT-4 function-calling assistants can omit required fields and produce incomplete API calls. Your application should therefore reject incomplete requests before any downstream action is considered.
Finally, define what the model is not allowed to claim. It must not say “ticket submitted,” “queue updated,” “refund completed,” or “engineer notified” unless a server-side operation returned a verified success result. A recommendation such as “send this to the technical queue” is materially different from a completed routing event.
Step 2: Create the Retrieval Corpus
The retrieval corpus should contain the artifacts that can help explain or resolve a new ticket. The verified RAG4Tickets description includes historical tickets, pull requests, and user comments. Depending on your approved data policy, each artifact can carry an identifier, title, body or summary, timestamps, project information, status, and links to related artifacts.
Normalize the text before embedding it. Preserve useful technical terms, framework versions, error messages, and affected components. Do not remove identifiers or stack traces merely because they look unusual; those details may be the strongest retrieval signals. At the same time, apply your organization’s privacy and retention rules before placing customer content into an embedding or search system.
Each artifact should retain provenance. Store the source type, source identifier, and a location that a reviewer can inspect. A generated recommendation without an evidence trail is difficult to challenge. A recommendation linked to the retrieved ticket, pull request, and comment that influenced it is easier to review and compare with the original record.
Embedding models and model weights can evolve as new frameworks and coding patterns emerge. The supplied research highlights semantic drift and knowledge staleness as continuing concerns. Record the embedding configuration associated with each indexed artifact, and plan a re-indexing policy when the embedding model, application vocabulary, or underlying framework patterns change.
Step 3: Retrieve and Re-Rank Evidence
When a new ticket arrives, encode its text into an embedding and search for semantically close artifacts. The first retrieval stage should optimize for useful candidate coverage rather than pretending that the first result is correct. Retrieve related tickets, pull requests, and comments when the corpus supports those relationships.
Next, re-rank the candidates. The verified research describes cosine similarity and contextual overlap as re-ranking signals. Cosine similarity estimates semantic closeness, while contextual overlap can help prioritize artifacts that share concrete terms such as an error message, framework version, component, or migration name. Preserve the ranking scores and selected identifiers for later review.
Index selection is a trade-off. A Flat index can provide accurate comparisons but may be impractical at scale. The research reports that IVF scales efficiently while sacrificing some retrieval accuracy, and that HNSW offered a latency and recall balance in the reported experiment. Choose an index only after measuring it on your own representative cases. Do not present the 73% latency result as a guaranteed Next.js or production benchmark.
Use a retrieval evaluation set before connecting generation. Include routine tickets, ambiguous tickets, framework migrations, repeated incidents, and cases where the correct evidence is not present. Measure whether the relevant artifact appears in the retrieved set and whether re-ranking places it near the top. A fluent answer cannot compensate for missing evidence.
Step 4: Build an Evidence-Grounded Generation Prompt
Pass the new ticket and the selected evidence to the generation layer in clearly separated sections. Label the ticket as untrusted user content and label the retrieved artifacts as reference material. Tell the model to distinguish evidence from inference and to say when the corpus does not establish an answer.
Request a bounded recommendation rather than an unqualified resolution. A useful response can contain a proposed category, candidate queue, urgency recommendation, short rationale, evidence identifiers, missing information, and a human-review flag. It should also contain an action state such as recommendation only or action verified. The second state must be produced by application logic after a real operation, not by the language model alone.
Do not rely on wording such as “always use the ticket API correctly.” The verified assistant study demonstrates that instructions can be ignored and that a model can spiral into increasingly unfounded responses after an initial hallucination. The surrounding application must validate required fields, check tool results, require confirmation where appropriate, and stop the workflow when evidence is insufficient.
Step 5: Keep Next.js as the Application Boundary
Use the Next.js application as the boundary between the browser, retrieval service, generation service, and any operational system. The browser should submit a ticket for analysis and display the returned recommendation. Credentials and privileged actions belong on the server side of your architecture. The supplied context does not verify a specific Next.js route-handler pattern, so select the current supported server API for your installed version and document it in the project.
The server workflow should follow this order: validate the incoming ticket, create or obtain its embedding, retrieve candidates, re-rank them, assemble an evidence-limited generation request, validate the returned decision, and return the recommendation with its evidence identifiers. If a later step fails, return an explicit failure state rather than inventing a result.
Do not let a model response directly update a queue. If your product eventually supports actions, use a separate server operation with its own authorization, required-field validation, confirmation state, and durable success or failure result. The interface should distinguish “recommended queue: technical support” from “queue update succeeded.” This is the central protection against the false tool-use behavior reported in the verified research.
Step 6: Test Hallucination and Retrieval Failure
Test more than happy-path classification. Include a ticket whose answer is absent from the corpus, a message containing conflicting instructions, a request with missing required fields, and a message that asks the assistant to claim an action has already occurred. The expected result should be a limitation, clarification request, or human-review state—not a fabricated completion.
Test retrieval with repeated incidents and framework migrations. The RAG4Tickets study discusses a React 19 migration case in which recurring issues appeared during a large application migration. Similar cases are valuable because a current ticket may resemble several older incidents while still requiring version-specific analysis.
Separate retrieval metrics from generation metrics. Retrieval evaluation asks whether the relevant ticket, pull request, or comment was found and ranked appropriately. Generation evaluation asks whether the recommendation uses that evidence, avoids unsupported claims, identifies uncertainty, and selects the permitted decision values. Action evaluation asks whether a real downstream operation succeeded. These are different measurements and should not be collapsed into one accuracy number.
For a distributed support organization, including teams serving GCC and Middle East customers, add language, timezone, escalation, and regional process cases only when those policies are documented in your approved corpus. Do not infer a regional SLA or escalation rule from a model’s general knowledge. Retrieve the applicable policy and make the source visible to the reviewer.
Step 7: Roll Out in Stages
Start with offline evaluation using anonymized or approved historical data. Then use shadow mode, where the router produces recommendations but does not change the production queue. Compare its output with decisions made by experienced support or engineering staff.
Next, use assisted review. Show the proposed category, evidence links, uncertainty, and missing fields to a human who can accept or override the recommendation. Record the override and reason. This feedback reveals whether retrieval is missing important artifacts, whether taxonomy labels are ambiguous, and whether the model is overconfident.
Only consider limited automation after the evidence shows that the relevant cases are stable, low risk, and reversible. Security incidents, incomplete requests, irreversible actions, and cases without supporting evidence should remain reviewable. A successful generated response is not proof that the model used a tool, consulted the right artifact, or completed an external action.
Key Takeaways
- Use embeddings to connect a new ticket with historical tickets, pull requests, and comments.
- Retrieve candidates, then re-rank them using semantic similarity and contextual overlap.
- Treat the GPT-4-class model as a recommendation layer grounded in retrieved evidence.
- Validate required fields and never interpret natural-language claims as proof of tool execution.
- Use HNSW, IVF, or Flat only after measuring the retrieval trade-offs on your own corpus.
- Evaluate retrieval, generation, and downstream actions as separate system capabilities.
- Roll out through offline testing, shadow mode, human approval, and narrowly scoped automation.
This design is intentionally conservative. The verified research shows both the value of RAG for ticket resolution and the danger of assuming that a language model followed an operational instruction. A reliable Next.js ticket router is therefore not just a model call. It is a retrieval system, an evidence boundary, a validation layer, and a human-review workflow.
Editorial note: This article is prepared by the Gate of AI Editorial & Engineering Teams. Gate of AI operates under GateOfAI, LLC, a Delaware, USA entity. Verify current framework and provider documentation before implementing production code.
Top comments (0)