An LLM Is Not an Application: What I’m Learning About Building GenAI with Backend Engineering

As I continue learning Forward Deployed Engineering and Generative AI, I am beginning to look at AI from a software engineer's perspective.

My third lecture focused on a distinction that seems simple at first:

An LLM is not the same thing as a GenAI application.

An LLM provides language capabilities. But a real application needs software around the model to make those capabilities useful.

The model is only one part of the system

Imagine a customer support team wants to summarize incoming support tickets.

The LLM can generate a summary. But someone still needs to build the system that receives the ticket, sends the right information to the model, handles the response, and makes that result available to the support team.

The overall workflow might look like this:

Support Ticket
      ↓
Backend Application
      ↓
LLM Provider API
      ↓
Generated Summary
      ↓
Backend Application
      ↓
Support Team

The model is important, but it is only one component in the workflow.

What if the model needs information it doesn't have?

Consider a user asking:

“Where is my order right now?”

A model cannot automatically know the user's latest order status. The application needs to retrieve that information from the relevant business system.

The application may need to:

1. Identify the requested information.
2. Retrieve the appropriate order data.
3. Provide the relevant context to the model.
4. Generate a clear response.
5. Return the result to the user.

This changes how I think about building AI features.

Instead of asking only, “Which model should I use?”, I also need to ask, “What information does this workflow require, and how will my application obtain it?”

Connecting GenAI to backend engineering

As a Java backend developer, I already understand APIs, services, controllers, external integrations, and data flow.

Those concepts still matter when integrating an LLM.

For example, a Spring Boot application can expose a REST endpoint, accept a support ticket, pass it to a service, and use Spring AI's ChatClient to communicate with a supported model provider.

The model interaction becomes part of an ordinary application workflow.

That does not mean the integration is automatically production-ready. Engineers still need to consider validation, errors, credentials, latency, token usage, and observability.

Why abstractions matter

It is possible to call an LLM provider directly through HTTP. A framework such as Spring AI provides another way to organize those interactions.

As requirements grow to include streaming, structured outputs, tool calling, retrieval, or multiple providers, an abstraction may help reduce provider-specific code in application logic.

The right choice depends on the application. Adding a framework without a real need can create unnecessary complexity, just as it can in other areas of software engineering.

One more design question

My lecture also raised an interesting question: should application instructions and user-provided data be treated as the same thing?

For example:

• Instruction: “Summarize this support ticket in two sentences.”
• Data: “Payment was deducted, but my order is still processing.”

These serve different purposes. Keeping them conceptually separate is a useful starting point when designing model interactions.

What I'm taking away

My understanding of GenAI is moving from the model itself toward the system built around it.

The model generates language. The application supplies context, connects external capabilities, handles the result, and integrates the model into a real workflow.

This is where my backend engineering experience becomes relevant to the AI systems I want to build.

An LLM can be powerful, but engineering the surrounding application is what turns that capability into a useful feature.

I'm continuing to explore the connection between backend engineering, system design, and production AI applications as part of my Forward Deployed Engineering learning journey.