DEV Community: Juan M. Altamirano

What Backend Engineers Get Wrong About AI Integration

Juan M. Altamirano — Mon, 13 Apr 2026 17:26:14 +0000

So you've been asked to "add AI" to your project. Maybe it's a chatbot, maybe it's a smart search, maybe it's just your PM watching too many YouTube videos about agents. Either way, here you are.

And look, as backend engineers, we're actually well-positioned for this. We know how to design APIs, handle async flows, think about failure modes. The problem is that LLMs don't behave like anything we've integrated before, and our instincts sometimes work against us.

Here, I will try to cover eight common mistakes that I've seen (and made).

1. Treating the LLM like a deterministic function

This is the big one.

We're used to thinking in terms of: same input → same output. Call a database, get a row. Call an API, get a JSON response. Write a unit test, pin the behavior forever.

LLMs don't work like that. Under the hood, they're predicting the next most probable token based on everything that came before, which means there's inherent randomness added into every response. Same prompt, different temperature, different day, and you might get a slightly different answer. Or a wildly different one.

Think of it less like calling a function and more like asking a very smart contractor to do some work for you. They'll usually get it right, but you need to review the output, define what "right" looks like, and have a plan for when they hand you something unexpected.

The practical consequence: don't build flows that assume the model always returns what you expect. Validate the output. Have fallbacks. Don't let a malformed response crash your service.

2. Not handling retries and timeouts

If you call an external REST API and it times out, you probably have retry logic. Maybe exponential backoff. Maybe a circuit breaker.

For some reason, when we start calling LLM APIs, all of that discipline disappears.

LLM calls are slow. We're talking in the order of seconds depending on the model and the response length. And they fail. Rate limits, provider outages, network issues, it all happens.

Set a timeout. Retry with backoff. If you're building something user-facing, stream the response so it doesn't feel like the app is frozen. Treat the LLM provider like what it is: an external dependency that can and will go down.

3. Ignoring token costs in loops

Token costs have a way of sneaking up on you.

A common pattern that seems harmless: you have a list of items, you loop over them, you call the LLM for each one. 50 items? 50 calls. Each one dragging along the same massive system prompt like dead weight. The model isn't doing 50x the thinking. You're just paying 50x for the setup.

It's like printing the entire company handbook before every meeting just to discuss one item.

Some things to think about:

Can you batch items into a single prompt? Sometimes yes, sometimes the quality drops.
Is the system prompt really 2000 tokens? Can it be 400?
Are you sending context the model doesn't need for that specific call?
Could you cache responses for identical or near-identical inputs?

Token costs are easy to ignore in development (small dataset, free tier) and painful to discover in production.

4. Breaking prompt caching without realizing it

This one is especially relevant if you're building agents.

Most providers cache prompts automatically and give you significant discounts when a request matches a previously seen prefix. In an agent loop where you're continuously resending the system prompt, full conversation history and tool calls, that cache is what keeps your bill reasonable.

The catch: think of it like a prefix match. The moment something differs from the previous call, the cache stops there and you pay full price from that point on.

A classic example is injecting the current timestamp into your system prompt. Seems harmless, but since it changes on every call, nothing after it ever gets cached, and you're paying full price every time.

Treat everything before the dynamic part of your prompt as sacred. Same order, same content, same whitespace. Push dynamic context as far down the message history as possible, after the static parts that can be cached.

This leads to decisions that look counterintuitive: sometimes you deliberately send more tokens just to keep the cacheable prefix intact. A 2000-token cached call is cheaper than a 1500-token cache miss. Once you internalize that, it changes how you structure your payloads entirely.

5. Treating prompt engineering as someone else's problem

"The AI team handles the prompts."

If you're building the integration, you need to understand how prompts work. Not at a research level, but enough to know why your endpoint is returning garbage.

The prompt is part of your application logic. It deserves the same attention as your SQL queries or your request validation. A bad prompt will produce bad output no matter how clean your surrounding code is.

At minimum, know the difference between system and user messages, understand what temperature does to your outputs, and learn when to use structured outputs, instead of trying to parse free-form text. Which brings me to...

6. Trusting free-form text when you don’t have to

I've seen code like this in the wild:

response = llm.complete("Analyze this and return: risk level, explanation, recommendation")
lines = response.split("\n")
risk = lines[0].split(":")[1].strip()

Please don't do this.

Modern LLM APIs support structured outputs — you define a schema, the model returns valid JSON that matches it. Every time. No parsing, no IndexError because the model decided to add an extra line.

In Python, pair it with Pydantic. In Typescript, use Zod for validation and parsing. In Go, define a struct and unmarshal directly. Your future self will thank you.

class AnalysisResult(BaseModel):
    risk_level: Literal["HIGH", "MEDIUM", "LOW", "NONE"]
    explanation: str
    recommendation: str

result = llm.complete_structured(prompt, schema=AnalysisResult)
# result.risk_level is always there. Always a string. Always one of the four values.

7. Putting everything in the context window and calling it RAG

RAG (Retrieval Augmented Generation) is genuinely useful. The idea is simple: instead of stuffing all your knowledge into the prompt, you retrieve only the relevant bits and include those.

But a lot of implementations I've seen are just... dumping documents into the prompt and calling it RAG.

Real RAG has a retrieval step. You embed the user's query, find the most semantically similar chunks from your knowledge base, and only send those. If you're sending 50 pages of documentation on every call, you're not doing RAG, you're doing expensive copy-paste.

The retrieval quality matters as much as the generation. Bad retrieval means irrelevant context, and irrelevant context means bad answers (no matter how good your model is).

8. Not thinking about what happens when the model is wrong

LLMs hallucinate. They confidently state things that are false. This isn't a bug that will get patched, it's just how these models work.

So the question isn't "what if the model is wrong?"... it's "what happens in my system when the model is wrong?"

If you're using an LLM to triage support tickets, a wrong answer means someone spends extra time investigating. Annoying, not catastrophic. But if you're using it to generate financial reports or medical summaries, wrong answers have a very different impact.

Design your system with the assumption that the model will sometimes be wrong. Add human review steps where it matters. Log the inputs and outputs so you can audit. Don't pipe raw LLM output into anything irreversible.

Wrapping up

None of this means LLMs aren't useful, they absolutely are. But they're a new kind of dependency with failure modes we're not used to. The good news is that most of the solid engineering practices we already have (validation, retries, observability, defensive design) apply here too. We just need to remember to actually use them.

One last thing: this space moves fast. What's a workaround today might be a built-in feature tomorrow. If you're already the kind of engineer who keeps up with tech, just make sure AI topics are in the mix. It's worth it.

If you've run into other footguns that aren't on this list, drop them in the comments. I'm curious what patterns other backend engineers have hit.

Stop Wasting Time on CVEs That Don't Affect You

Juan M. Altamirano — Mon, 16 Mar 2026 18:47:53 +0000

The Problem

Aren't you tired of pushing new code and then a few days later receiving an alert from Github's Dependabot? Well, I am.
The most annoying part is looking for the CVE, reviewing your code and then detecting that you aren't using the affected part.
Rinse and repeat for every single alert.

The solution?

That's why I built dep_shield — a CLI that I can plug into my common workflow (lint -> dep_shield -> tests -> sonar) and get a straight answer: "this CVE affects you" or "relax, you're fine."

How dep_shield Works

The flow is straightforward:

Parse dependencies — Read requirements.txt or pyproject.toml, extract packages and versions
Check for CVEs — Query the OSV database for known vulnerabilities
Find usage in code — Scan your Python files to see where you import vulnerable packages
AI-powered analysis — Send the CVE description + your import context to an LLM and ask: "Does this actually affect me?"

The Interesting Parts

Parsing Dependencies (Both Formats)

The tool supports requirements.txt and pyproject.toml — it figures out which one you're using.
Why both? Well, requirements.txt is still widely used, but uv is taking over fast.

Querying OSV (the free vulnerability database)

OSV over NVD or Snyk? No API key, no rate limits, no pricing tiers. NVD is slow and wants you to parse CPE identifiers. Snyk needs auth and has usage limits.
OSV just works — POST a package name, get vulnerabilities back. For a CLI tool meant to run locally and fast, that's exactly what I needed.

OSV_API_URL = "https://api.osv.dev/v1/query"

def query_vulnerabilities(package_name: str, version: str | None) -> list[Vulnerability]:
    payload = {
        "package": {
            "name": package_name,
            "ecosystem": "PyPI"
        }
    }
    if version:
        payload["version"] = version

    response = httpx.post(OSV_API_URL, json=payload, timeout=10.0)
    return parse_vulnerabilities(response.json())

{
    "vulns": [
        {
            "id": "GHSA-cpwx-vrp4-4pq7",
            "summary": "Jinja2 vulnerable to sandbox breakout through attr filter selecting format method",
            "details": "An oversight in how the Jinja sandboxed environment interacts with the `|attr` filter allows an attacker that controls the content of a template to execute arbitrary Python code.\n\nTo exploit the vulnerability, an attacker needs to control the content of a template. Whether that is the case depends on the type of application using Jinja. This vulnerability impacts users of applications which execute untrusted templates.\n\nJinja's sandbox does catch calls to `str.format` and ensures they don't escape the sandbox. However, it's possible to use the `|attr` filter to get a reference to a string's plain format method, bypassing the sandbox. After the fix, the `|attr` filter no longer bypasses the environment's attribute lookup.",
            "aliases": ["CVE-2025-27516"],
            "modified": "2026-02-04T04:14:58.595738Z",
            "published": "2025-03-05T20:40:14Z",
            "affected": [
                {
                    "package": { "name": "jinja2", "ecosystem": "PyPI", "purl": "pkg:pypi/jinja2" },
                    "ranges": [{ "type": "ECOSYSTEM", "events": [{ "introduced": "0" }, { "fixed": "3.1.6"}] }]
                }
            ]
        }
    ]
}

The full response has more fields (references, versions, etc.), but these are the relevant ones.

Finding Where You Actually Use the Package

It's easy to forget where you actually use a dependency — especially in large projects with dozens of packages. And not all usage is equal: importing requests in your core API is very different from importing it in a one-off migration script.

def scan_file_for_package(file_path: Path, package_name: str) -> list[CodeUsage]:
    pattern_import = rf"^import\s+{package_name}(\s|,|$|\.)"
    pattern_from = rf"^from\s+{package_name}(\s|\.)"
    result = []

    with open(file_path, 'r', encoding='utf-8', errors='ignore') as file:
        for line_num, line in enumerate(file, 1):
            line = line.strip()
            if line.startswith("#"):
                continue

            if re.match(pattern_import, line):
                import_type = "import"
            elif re.match(pattern_from, line):
                import_type = "from"
            else:
                continue

            result.append(CodeUsage(
                file_path=str(file_path),
                line_number=line_num,
                line_content=line,
                import_type=import_type
            ))
    return result

The AI Part: Asking A Model If It Matters

Now, here's where it gets interesting. I send the LLM:

The CVE description
The import statements from your code

And ask: "Given how this code uses the package, does this vulnerability apply?"

Important: Only import lines are sent to the LLM — not your actual business logic. The model sees from requests import Session, not the body of your functions. Your code stays local.

_SYSTEM_PROMPT = """You are a security analyst reviewing Python dependency vulnerabilities.
You will be given a CVE description and import-level code evidence.
Your job is to determine if the vulnerability realistically affects this codebase.

Risk level definitions:
- HIGH: The imports directly expose the vulnerable code path
- MEDIUM: Package is imported but no clear evidence the vulnerable feature is used
- LOW: Package only imported in tests or dev tooling
- NONE: CVE conditions are not plausible in this codebase"""

I use Pydantic to force the model into a consistent format, no free-form text that I'd have to parse.
It either gives me a valid ImpactAnalysis or it fails. No "well, it depends..." answers.

class ImpactAnalysis(BaseModel):
    risk_level: Literal["HIGH", "MEDIUM", "LOW", "NONE"]
    explanation: str
    recommendation: str

Making It Smarter Over Time (RAG)

Here's the thing — the LLM doesn't know your codebase. It analyzes each CVE in isolation. But what if it could remember past analyses?

That's where ChromaDB comes in. Every time the tool analyzes a vulnerability, it stores the result. Next time a similar CVE shows up (same package, similar vulnerability type), it retrieves past analyses as context.

def store_analysis(vulnerability_id: str, analysis: ImpactAnalysis, code_context: str):
    collection.add(
        documents=[f"{vulnerability_id}: {analysis.explanation}"],
        metadatas=[{"risk": analysis.risk_level, "package": package_name}],
        ids=[vulnerability_id]
    )

def get_similar_analyses(vulnerability_description: str, limit: int = 3):
    results = collection.query(
        query_texts=[vulnerability_description],
        n_results=limit
    )
    return results

The result? The tool gets better the more you use it. If you analyzed a requests CVE last month and a new one appears, the model sees how you handled it before.

What the Output Looks Like

What I Learned

Honestly? It reinforced an ancient rule: context is everything
A CVE that sounds critical in the abstract becomes irrelevant when you realize you only import that package in a test helper. OSV turned out to be the perfect data source — free, fast, no API keys, solid Python coverage. And while the LLM does a surprisingly good job at triage, I still treat its output as a suggestion, not a verdict. The RAG layer (ChromaDB) was an afterthought, but it's become one of the most useful parts: the tool genuinely gets better as it sees more of your codebase.

What's Next

I have a few ideas in mind:

More languages — JavaScript and Go are probably the next ones
CI/CD integration — run it in your pipeline, make it fail the build if there is any 'HIGH' impact
Offline mode — run it against a local LLM

Try It Yourself

Give it a shot and let me know what breaks. Seriously — I'd appreciate any feedback.
dep_shield

Internationalization: a practical example with Spring Boot

Juan M. Altamirano — Tue, 01 Feb 2022 13:05:05 +0000

If you started coding a long time ago in a not so far away galaxy, the idea of being globally connected and exposing your web services to the world was only a dream. And that’s why you probably wouldn't consider adding internationalization (I18N) to your service.

Over the years and with global exposure in mind, frameworks started to include tools to ease the addition of I18N in our APIs.

What is I18N

You may already know what I18N is, but for those of you who are thinking of taking a quick peek at Wikipedia to find out, here's a brief definition instead:

“Internationalization is the process of designing a software application so that it can be adapted to various languages and regions without engineering changes.”

A practical example: ResourceBundleMessageSource

Needless to say, each framework has its own way to tackle I18N.
In this post, we will see how easy it is to add I18N to our API, by reviewing a possible solution for Spring Boot.
Considering that the solution is very simple, I will show you examples in Java and Kotlin.

Overall process

The process consists of four steps:

Creating our message service
Adding a resource bundle for all the languages we want to support
Adding variables to the application properties
Start using our message service

1. Creating the service

Let's start by creating a class that will work as a message service.

Java

@Service
public class MessageService {

    @Autowired
    private ResourceBundleMessageSource messageSource;

    @Value("${custom.app.locale}")
    private String systemLanguage;

    private Locale locale;

    @PostConstruct
    private void init() {
        locale = new Locale(systemLanguage);
        LocaleContextHolder.setDefaultLocale(locale);
    }

    public String getMessage(String prefix, String key){
        return getMessage(prefix, key, "");
    }

    public String getMessage(String prefix, String key, String value){
        List<String> listOfValues = Collections.singletonList(value);
        return getMessage(prefix, key, listOfValues);
    }

    public String getMessage(String prefix, String key, List<String> args){
        return messageSource.getMessage(concatPrefixAndKey(prefix, key), args.toArray(), locale);
    }

    public String getMessage(String prefix, String key, List<String> args, Locale requestedLocale){
        if (requestedLocale == null){
            return getMessage(prefix, key, args);
        } else {
            return messageSource.getMessage(concatPrefixAndKey(prefix, key), args.toArray(), requestedLocale);
        }
    }

    public String getRequestLocalizedMessage(String prefix, String key){
        return getRequestLocalizedMessage(prefix, key, new ArrayList<>());
    }

    public String getRequestLocalizedMessage(String prefix, String key, List<String> args){
        return getMessage(prefix, key, args, LocaleContextHolder.getLocale());
    }

    private String concatPrefixAndKey(String prefix, String key){
        return prefix.concat(".").concat(key);
    }
}

Kotlin

@Service
class MessageService {
    @Autowired
    private lateinit var messageSource: ResourceBundleMessageSource

    @Value("\${app.locale}")
    private lateinit var localLanguage: String

    private lateinit var locale: Locale

    @PostConstruct
    fun init() {
        locale = Locale(localLanguage)
        LocaleContextHolder.setDefaultLocale(locale)
    }

    fun getMessage(prefix: String, key: String): String {
        return getMessage(prefix, key, emptyList())
    }

    fun getMessage(prefix: String, key: String, value: String): String {
        return getMessage(prefix, key, listOf(value))
    }

    fun getMessage(prefix: String, key: String, args: List<String>): String {
        return messageSource.getMessage("$prefix.$key", args.toTypedArray(), locale)
    }

    fun getMessage(prefix: String, key: String, args: List<String>, locale: Locale?): String {
        return locale?.let {
            messageSource.getMessage("$prefix.$key", args.toTypedArray(), locale)
        } ?: getMessage(prefix, key, args)
    }

    fun getRequestLocalizedMessage(prefix: String, key: String): String {
        return getRequestLocalizedMessage(prefix, key, emptyList())
    }

    fun getRequestLocalizedMessage(prefix: String, key: String, args: List<String>): String {
        return getMessage(prefix, key, args, LocaleContextHolder.getLocale())
    }
}

2. Adding the messages

Now, we will create a resource bundle inside our resources folder and we will add a properties file for each language we want to support.

messages.properties

## GENERIC MESSAGES
generic.message.i18n=Hey, this is a message in english.

messages_en.properties

## GENERIC MESSAGES
generic.message.i18n=Hey, this is a message in english.

messages_es.properties

## GENERIC MESSAGES
generic.message.i18n=Hey, este es un mensaje en español.

messages_fr.properties

## GENERIC MESSAGES
generic.message.i18n=Salut, c'est un message en français.

3. Updating the application.properties

Here we will add two variables, one for the default language and one for the application encoding.

# The default locale for our API
custom.app.locale=fr

# This sets the Spring boot encoding used for the API responses
spring.messages.encoding=ISO-8859-1

4. Using the service

And the last step, we need to wire our message service and use it. For this, I created a simple controller that calls the getRequestLocalizedMessage method.
The method uses the LocalContextHolder to detect the Accept-Language header. If the header is present, it returns the response in the requested language. If not, it returns the message in the default language.

Java

@RestController
@RequestMapping("/test")
public class TestController {

    @Autowired
    private MessageService messageService;

    private final String prefixKey = "generic.message";

    @GetMapping(produces = {MediaType.APPLICATION_JSON_VALUE})
    public ResponseEntity<String> getTest(){
        return ResponseEntity.status(HttpStatus.OK).body(messageService.getRequestLocalizedMessage(prefixKey, "i18n"));
    }
}

Kotlin

@RestController
@RequestMapping("/test")
class TestController {

    @Autowired
    private lateinit var messageService: MessageService

    private val prefixKey = "generic.message"

    @GetMapping(produces = [MediaType.APPLICATION_JSON_VALUE])
    fun getTest(): ResponseEntity<String> = ResponseEntity
        .status(HttpStatus.OK)
        .body(messageService.getRequestLocalizedMessage(prefixKey, "i18n"))
}

Alternatively, you can use the other methods in the MessageService to achieve the same goal.

// You can pass the Locale as an argument to the getMessage function to get the message in a specific language

Locale requestedLocale = new Locale("es");
messageService.getMessage(prefixKey, "i18n", new ArrayList<>(), requestedLocale);

// OR

// You can use the getMessage without a specific locale to get the message in the default language
messageService.getMessage(prefixKey, "i18n");

Final comments

As we’ve seen in this post, adding I18N to our API is very simple. So, if you are thinking about creating APIs to be globally consumed (and if you are not already doing it), it is not a bad idea to add this to your other good practices (clean code, documentation, testing…).

If you are working on the JVM platform and using Spring Boot in your projects, I hope this post was helpful.
If you are working with other frameworks/languages, don’t worry, the internet is full of examples and information that will help you to implement it in your project.