DEV Community: Nikhil Kamani

My AI Chatbot Started "Faking" Tool Calls in Plain Text. Here's Why.

Nikhil Kamani — Sun, 16 Aug 2026 06:10:03 +0000

I built a small AI "digital twin": a chatbot that answers questions about my background the way I would, and calls a tool to notify me the moment someone wants to get in touch.
Stack is Groq's API running llama-3.3-70b-versatile, Gradio for the UI, deployed on Hugging Face Spaces.

I skipped LangChain and any agent framework on purpose. I wanted to actually watch an agent decide when and how to use a tool, not have a framework make that decision invisible to me.

That choice is what taught me the most, not about building a chatbot, but about how an LLM actually behaves once you hand it a tool and just tell it "use this when needed."

The bug

I gave the model a record_user_details tool: if a visitor wants to be contacted, the model calls it with their email, and I get a push notification. Simple enough.

Except every so often, instead of triggering the tool, the model would just print this straight into the chat reply:

<function=record_user_details>{"email": "someone@example.com", "notes": "wants to connect"}</function>

Visible to the visitor, as plain text. And since it never actually went through the API's tool-call mechanism, my code never saw it. No notification. No record. The model was "calling" a function that, as far as my backend was concerned, didn't happen.

Why it happens

Llama 3.x models were trained on two different ways to represent a tool call:
the structured tool_calls field an API returns and a text-based invocation syntax using special tags. Depending on prompt shape and how the serving layer steers the model, it sometimes reaches for the second one instead of the first, especially under ambiguous instructions about when a tool should fire at all.

This isn't a Groq-specific bug, it's a known quirk of the model itself. I only found this by staring at raw completions in my logs instead of trusting the SDK's parsed response, then cross-checking against the model card and a few GitHub issues describing the exact same symptom.

The fix (three layers, because one alone wasn't enough)

temperature=0. Doesn't eliminate the behavior, but makes the model far more consistent about which format it reaches for.
Tightened the tool description. I was vague about when to call the tool. Being explicit about the trigger condition, and explicit that it should never be described in the reply text, cut down the ambiguous cases where the model improvised.
Defensive parsing. This is the one that actually matters for a real visitor. Even with the first two fixes, I don't fully control the model. So I wrapped the response handling to detect a malformed, text-only "call" and fail soft: a polite "could you rephrase that?" instead of leaking a raw function tag into someone's chat window.

What this taught me about agent behavior

The model isn't "deciding" to call a tool the way the word "agent" makes it sound. It's predicting the next most likely tokens, and sometimes the most likely tokens look like a tool call without ever going through the real mechanism.
A framework would have smoothed that over for me automatically. Hand-rolling it meant I actually saw, first-hand, how much less deterministic "agent behavior" is than the term implies.

Try it: https://huggingface.co/spaces/nik1988/digital-twin

(Note: it's on a free Hugging Face Space, so it may have gone to sleep from inactivity. If it doesn't load right away, hit "Restart Space" and give it a few seconds to spin back up.)

Curious about two things from anyone who's built something similar:

If you were extending a digital-twin chatbot like this, what would you add next?
I've been going back and forth on whether it's worth more tooling (calendar booking, resume Q&A grounded in a vector store) or keeping it deliberately minimal.

Drop your ideas below, genuinely want to see what other people have tried with this.

AI can write the code. It can't tell you why it's correct under load.

Nikhil Kamani — Sat, 08 Aug 2026 04:09:25 +0000

A junior developer asked me last week: "AI writes half my code now. Why does the interview process still grill me on HashMap internals and thread safety?"

I've interviewed 300+ Java developers over 13 years. Here's what I told him.

Ask any AI assistant to write a counter, a cache, or a payment method, and it'll hand you something that compiles, runs, and looks completely reasonable. It will also, quite often, hand you code like this:

private volatile int count = 0;

public void increment() {
    count++;
}

This looks fine. It passes every test you'd normally write. It will also silently lose updates the moment two threads call increment() at the same time — because volatile guarantees visibility, not atomicity, and count++ is three separate operations under the hood. AI doesn't know that this specific line is dangerous, because syntactically there's nothing wrong with it. It only becomes a bug under concurrent load, in production, weeks after it shipped.

When I ask a candidate "walk me through what happens when two threads call this method at the same time," I'm not testing whether they can write a counter. I'm testing whether they can catch this exact bug in a code review — including one AI just generated for them.

The fundamentals didn't get less important. They got more important.

A few years ago, the bottleneck was typing out correct code. Now AI removes most of that friction — which means the bottleneck has moved entirely to judgment: can you look at generated code and know whether it's actually safe?

That skill doesn't come from prompting well. It comes from actually understanding what's happening underneath

What actually changed in interviews

AI changed what junior developers are asked to produce day to day. It didn't change what interviewers are testing for. Every interview I run this year still comes back to the same core question: does this person understand memory, concurrency, and data structures well enough to catch the bug — or do they just know how to get an AI to write code that compiles?

If anything, this makes fundamentals a bigger differentiator between candidates than they were five years ago, not a smaller one. Everyone can produce working-looking code now. Far fewer people can tell you why it's wrong.

I put together everything I actually ask candidates into a guide — 700+ pages: Core Java, Concurrency, Spring Boot, Design Patterns, Coding Round Patterns.

Full guide: https://kamaninikhil.gumroad.com/l/java-interview-guide

Feedback or questions? Email me: kamaninikhil71@gmail.com

12 Coding Pattern Scenario Questions That Expose Real Gaps

Nikhil Kamani — Thu, 30 Jul 2026 02:52:14 +0000

I have 13 years of Java experience and have interviewed hundreds of developers at MNCs. Most candidates know the 14 major coding patterns by name. Fewer can look at a fresh problem and instantly know which one fits — that's the actual skill being tested. Here's the mapping.

Find if a sorted array has two numbers that sum to a target, without extra space → Two Pointers
Find the longest substring without repeating characters → Sliding Window
Group a list of words into anagram sets → Hash Map / Frequency Counter
Answer many "sum between index i and j" queries efficiently, without recalculating each time → Prefix Sum
Find an element in a rotated sorted array in O(log n) → Binary Search and Variants
Check if a string of brackets is balanced → Stack-Based Problems
Find the maximum sum of any contiguous subarray → Greedy / Kadane's Algorithm
Find the shortest path between two nodes in an unweighted graph → Tree/Graph Traversal (BFS)
Find the maximum value of items that fit in a knapsack of limited capacity → Dynamic Programming (Pick / No-Pick)
Find the longest common subsequence between two strings → Dynamic Programming (Sequence Matching)
Generate all valid combinations of balanced parentheses → Backtracking
Find the Kth largest element in an unsorted array without fully sorting it → Quick Select

I put together a complete backend interview guide covering Core Java, Java 8-21, Multithreading, Spring Boot, Microservices, Design Patterns, and Coding Round Patterns.

Full guide: https://kamaninikhil.gumroad.com/l/java-interview-guide

Feedback or questions? Email me: kamaninikhil71@gmail.com

Bookmark this — which one do you always blank on when you're under pressure?

What Actually Happens When the 201st User Hits Your Spring Boot App

Nikhil Kamani — Sat, 25 Jul 2026 04:38:20 +0000

The setup
Your Spring Boot app runs on embedded Tomcat. By default, Tomcat has a pool of about 200 threads. Every incoming HTTP request grabs one thread from that pool, holds it for the entire duration of the request, and returns it when the response is sent.

Your typical controller does something like this:

@GetMapping("/order/{id}")
public OrderView getOrder(@PathVariable String id) {
    var order    = orderClient.fetch(id);      // REST call — ~200ms waiting
    var payment  = paymentClient.status(id);   // REST call — ~200ms waiting
    var shipping = shippingClient.track(id);   // REST call — ~200ms waiting
    return combine(order, payment, shipping);
}

Three sequential network calls. About 600ms per request — and here's the key part: for almost all of those 600ms, the thread is doing nothing. It's just parked, waiting for other servers to reply.

The 201st user
200 users hit this endpoint at the same time. All 200 Tomcat threads are now holding a request, each one blocked, each one waiting on a slow downstream call.

The 201st user arrives.

There is no thread left to give them. Their request doesn't fail — it queues. It sits and waits for one of the 200 threads to finish and free up. Your server's CPU is nearly idle (everyone's just waiting on I/O), your memory is fine — and yet users are stuck in a line. This is thread pool exhaustion, and it's one of the most common ways a healthy-looking service falls over under load.

For years we solved this two painful ways: crank the pool size way up (each platform thread costs ~1MB of memory, so this doesn't scale far), or rewrite everything in reactive/WebFlux style (fast, but the code gets much harder to read and debug).

What virtual threads change

A platform thread is a thin wrapper around a real OS thread — expensive, ~1MB, you can have a few thousand. A virtual thread is managed by the JVM instead of the OS. It costs a few hundred bytes, and you can have millions of them.

The magic is what happens when a virtual thread blocks on I/O. Instead of sitting there holding an OS thread hostage, the JVM unmounts it — the virtual thread steps aside, the real OS thread is freed to go run someone else's work, and when the network reply finally comes back, the virtual thread gets re-mounted onto any free OS thread to continue.

So during those three REST calls, the underlying OS thread isn't wasted waiting. It's off serving other requests.

In Spring Boot 3.2+, you turn this on with a single line:

spring.threads.virtual.enabled=true

Now every request runs on its own fresh virtual thread. There is no pool of 200 to exhaust. The 201st user — and the 20,000th — each get their own virtual thread, and the handful of real OS threads underneath stay busy instead of blocked. Same simple blocking code above. No WebFlux. No callback spaghetti.

The catch (this is the real interview follow-up)

Virtual threads only help when threads spend their time waiting — I/O-bound work. For CPU-bound work (heavy computation), there's nothing to unmount from, so they give no benefit over a normal pool sized to your cores.

And one trap that bites people: if a virtual thread blocks inside a synchronised block, the JVM currently can't unmount it — it stays "pinned" to its OS thread for the whole block, and you lose the entire advantage. In code that runs on virtual threads, prefer ReentrantLock over synchronised for anything that wraps a blocking call.

That's the full answer to "what happens to the 201st user" — the pool, the exhaustion, the unmount/re-mount, and the one line that changes it. If you can tell that story in an interview, you're already ahead of most candidates.

I cover this and more (Core Java, Java 8 to 21, Multithreading, Spring Boot, Microservices, Design Patterns, Coding Round Patterns) in my guide.

Full guide: https://kamaninikhil.gumroad.com/l/java-interview-guide

Feedback or questions? Email me: kamaninikhil71@gmail.com

Ever been burned by thread pool exhaustion in production? What did you do before virtual threads existed?

2 Concurrency Traps That Look Thread-Safe But Aren't

Nikhil Kamani — Mon, 20 Jul 2026 03:45:42 +0000

I have 13 years of Java experience and have interviewed hundreds of developers at MNCs. Both of these come up constantly in multithreading rounds — and both involve code that looks perfectly safe until you think about it for another 10 seconds.

Q1: Is this thread-safe?

private volatile int count = 0;

public void increment() {
    count++;
}

Answer: No.

Reason, in plain terms: volatile guarantees visibility — every read and write goes straight to main memory, so all threads see the latest value immediately. It does not guarantee atomicity. count++ is actually three separate steps: read the value, add 1, write it back. volatile makes each of those three steps visible to other threads, but it doesn't stop another thread from jumping in between them.

Thread-1: reads count (0)
Thread-2: reads count (0)      <- interleaves here
Thread-1: writes count = 1
Thread-2: writes count = 1     <- overwrites Thread-1's update

Result: count = 1 (should be 2)

This is why a "thread-safe" counter silently loses updates under load, and it's almost impossible to catch in testing — it only shows up under real concurrent traffic.

Fix: use AtomicInteger, which provides both visibility and atomicity:

private final AtomicInteger count = new AtomicInteger(0);

public void increment() {
    count.incrementAndGet();
}

Q2: Is this safe on a `ConcurrentHashMap`?

ConcurrentHashMap<String, Integer> map = new ConcurrentHashMap<>();

if (!map.containsKey("user1")) {
    map.put("user1", 1);
}

Answer: No.

Reason: every individual operation on ConcurrentHashMap is thread-safe — containsKey() alone is safe, put() alone is safe. But the sequence of the two together is not one atomic unit. Another thread can insert "user1" in the gap between your containsKey() check and your put() call, and you'll silently overwrite it.

Fix: use the map's built-in atomic compound methods instead of check-then-act:

map.putIfAbsent("user1", 1);                        // atomic check-and-put
map.compute("key", (k, v) -> v == null ? 1 : v + 1); // atomic get-modify-put
map.merge("key", 1, Integer::sum);                   // atomic get-modify-put

I cover this and more (Core Java, Java 8 to 21, Multithreading, Spring Boot, Microservices, Design Patterns, Coding Round Patterns) in my guide.

Free sample: https://drive.google.com/file/d/1u3PQbTY1gLn34UmWG7Cxx4cmdibD2dvU/view?usp=sharing

Full guide: https://kamaninikhil.gumroad.com/l/java-interview-guide

Feedback, typos, or suggestions? Email me: kamaninikhil71@gmail.com

Which one would you have caught in a code review? Drop it in the comments.

Circuit Breakers Come Up In Almost Every Microservices Interview — Here's The 3-State Flow

Nikhil Kamani — Tue, 14 Jul 2026 03:13:23 +0000

I have 13 years of Java experience and have interviewed hundreds of developers at MNCs. "Explain circuit breaker state transitions" is one of the most common microservices questions I ask — most candidates can name the pattern but fumble the actual states.

Here they are:

1. CLOSED — Normal Operation

All requests go through to the real service. The circuit breaker silently tracks successes and failures in the background.

Moves to OPEN when: the failure rate crosses a threshold — e.g. 50% of the last 10 calls failed.

2. OPEN — Service Is Failing

Rejects ALL requests immediately. Doesn't even call the service — just returns a fallback response instantly.

Moves to HALF_OPEN when: a wait duration elapses — e.g. after 60 seconds, to give the service a chance to recover.

3. HALF_OPEN — Testing Recovery

Allows a small number of test requests through — e.g. 3 calls. Everything else still gets the fallback.

Test calls succeed → back to CLOSED. Test calls fail → back to OPEN.

Why it actually matters, not just theory:

Say Payment Service goes down.

Without a circuit breaker: 100 users each wait 30 seconds for a timeout. Threads pile up waiting on a dead service. The failure cascades into your own system.

With a circuit breaker: the first ~10 requests fail and the circuit OPENS. The next 90 requests fail instantly, no waiting. Users get "Payment down, try later" and the rest of the system stays responsive.

The code (Spring Boot + Resilience4j):

@Service
public class OrderService {

    @CircuitBreaker(name = "payment-service", fallbackMethod = "paymentFallback")
    public PaymentResult processPayment(Order order) {
        return paymentClient.charge(order.getTotal());
    }

    // Called when circuit is OPEN or the call fails
    private PaymentResult paymentFallback(Order order, Exception e) {
        return PaymentResult.failed("Try again later.");
    }
}

resilience4j:
  circuitbreaker:
    instances:
      payment-service:
        failure-rate-threshold: 50
        wait-duration-in-open-state: 60s

The follow-up I always ask next: how is this different from Retry? Retry helps with transient blips — a single dropped packet, a momentary glitch. Circuit breaker helps with sustained outages — there's no point retrying against a service that's been down for 2 minutes, you're just adding load to something already struggling.

Other resilience patterns worth knowing alongside this: Timeout (gives up waiting after a max duration), Bulkhead (isolates thread pools so one failing service can't starve others), and Rate Limiter (caps requests per second to protect a service).

I cover this and more (Core Java, Java 8 to 21, Multithreading, Spring Boot, Microservices, Design Patterns, Coding Round Patterns) in my guide.

Free sample: https://drive.google.com/file/d/1u3PQbTY1gLn34UmWG7Cxx4cmdibD2dvU/view?usp=sharing

Full guide: https://kamaninikhil.gumroad.com/l/java-interview-guide

Feedback, typos, or suggestions? Email me: kamaninikhil71@gmail.com

Which state trips people up most — OPEN or HALF_OPEN? Drop it in the comments.

12 Java Collections Scenario Questions That Expose Real Gaps

Nikhil Kamani — Sat, 11 Jul 2026 04:42:26 +0000

Most "explain HashMap" interview questions are dead now — anyone
can look that up. What separates candidates today is scenario
reasoning: given a real constraint, which collection do you reach
for, and why not the obvious one?

12 questions worth testing yourself on:

You need an LRU cache that evicts the least recently used entry
automatically when full. Which collection do you reach for?
A producer thread generates data faster than the consumer can
process it, and the app is running out of memory. What do you
change?
You need the top 10 highest scores out of 1 million incoming
values, without holding all 1 million in memory. How?
Multiple threads read a config map constantly, but writes
happen once a day. ConcurrentHashMap, synchronizedMap, or
something else entirely?
You store a mutable object as a HashMap key, then modify one
of its fields after inserting it. What breaks?
Every key in your HashMap somehow returns the same hashCode().
What happens to lookup performance, and why?
You remove an element from a List while iterating it with a
for-each loop. What exception do you get — and why does it happen
even in single-threaded code?
Why does iterator.remove() avoid the exception in Q7, but
list.remove() inside the same loop doesn't?
What's the real difference between a fail-fast and a fail-safe
iterator — and where do ArrayList, HashMap, ConcurrentHashMap, and
CopyOnWriteArrayList each fall?
You need a Set that stays sorted at all times and supports
range queries — "give me everything between X and Y." Which
collection, and why not just sort a HashSet every time you need it?
A teammate says LinkedList is always faster for insertion than
ArrayList. Is that actually true in production?
Your CopyOnWriteArrayList is quietly driving up memory and GC
pressure. Why — and when should you actually reach for it?

How many could you answer confidently without looking anything up?

I've written out full answers with code and the reasoning behind
each one in my guide, alongside Java 8-21, Multithreading, Spring
Boot, Microservices, Design Patterns, and Coding Round Patterns.

Free sample: https://drive.google.com/file/d/1u3PQbTY1gLn34UmWG7Cxx4cmdibD2dvU/view?usp=sharing

Full guide: https://kamaninikhil.gumroad.com/l/java-interview-guide

2 @Transactional Traps That Catch Even Senior Java Developers in interviews

Nikhil Kamani — Wed, 08 Jul 2026 03:52:46 +0000

I have 13 years of Java experience. I have also interviewed hundreds of developers at MNCs.

Two questions about @Transactional that catch even senior developers. Try answering before you scroll down.

Q1: If a non-transactional method calls a `@Transactional` method in the same class, does the transaction apply?

@Service
public class OrderService {

    public void placeOrder(Order order) {
        this.saveOrder(order);
    }

    @Transactional
    public void saveOrder(Order order) {
        orderRepository.save(order);
        inventoryRepository.update(order);
    }
}

Answer: No.

Reason, in plain terms: when you add @Transactional, Spring does not add the transaction logic inside your class. Instead, it creates a second object that sits in front of your class — think of it as a wrapper. Other classes that call saveOrder() (through @Autowired) actually call this wrapper first. The wrapper starts the transaction, then calls your real method, then commits or rolls back.

This wrapper only exists outside your class. So when your own code calls this.saveOrder(order), it skips the wrapper completely and goes straight to the real method. No wrapper means no transaction logic. No rollback if inventoryRepository.update() fails.

This bug never shows up in tests. It only shows up in production, when a failure leaves your data half-updated.

Fix: call the method from a different class, or inject the wrapper of your own class:

@Service
public class OrderService {

    @Autowired
    private OrderService self; // this is the wrapper, not the raw object

    public void placeOrder(Order order) {
        self.saveOrder(order); // now it goes through the wrapper
    }

    @Transactional
    public void saveOrder(Order order) {
        orderRepository.save(order);
        inventoryRepository.update(order);
    }
}

Q2: Does `@Transactional` roll back on every exception?

@Transactional
public void processRefund(Order order) throws RefundException {
    paymentRepository.markRefunded(order);
    externalRefundService.notify(order); // throws RefundException
}

Answer: No.

Reason: by default, @Transactional only rolls back on unchecked exceptions (RuntimeException and its subclasses). RefundException above is a checked exception. So if it is thrown, Spring lets the transaction commit anyway — including the markRefunded write that happened right before the failure.

Fix: tell Spring to roll back on any exception, not just unchecked ones:

@Transactional(rollbackFor = Exception.class)
public void processRefund(Order order) throws RefundException {
    paymentRepository.markRefunded(order);
    externalRefundService.notify(order);
}

I cover topics like this in more depth in my guide — Core Java, Java 8 to 21, Multithreading, Spring Boot, Microservices, Design Patterns, and Coding Round Patterns.

Free sample: https://drive.google.com/file/d/1u3PQbTY1gLn34UmWG7Cxx4cmdibD2dvU/view?usp=sharing

Full guide: https://kamaninikhil.gumroad.com/l/java-interview-guide

Got either one right? Drop it in the comments.

DEV Community: Nikhil Kamani

My AI Chatbot Started "Faking" Tool Calls in Plain Text. Here's Why.

The bug

Why it happens

The fix (three layers, because one alone wasn't enough)

AI can write the code. It can't tell you why it's correct under load.

12 Coding Pattern Scenario Questions That Expose Real Gaps

What Actually Happens When the 201st User Hits Your Spring Boot App

2 Concurrency Traps That Look Thread-Safe But Aren't

Q1: Is this thread-safe?

Q2: Is this safe on a ConcurrentHashMap?

Circuit Breakers Come Up In Almost Every Microservices Interview — Here's The 3-State Flow

1. CLOSED — Normal Operation

2. OPEN — Service Is Failing

3. HALF_OPEN — Testing Recovery

12 Java Collections Scenario Questions That Expose Real Gaps

2 @Transactional Traps That Catch Even Senior Java Developers in interviews

Q1: If a non-transactional method calls a @Transactional method in the same class, does the transaction apply?

Q2: Does @Transactional roll back on every exception?

Q2: Is this safe on a `ConcurrentHashMap`?

Q1: If a non-transactional method calls a `@Transactional` method in the same class, does the transaction apply?

Q2: Does `@Transactional` roll back on every exception?