Native Inference: Testing an SLM (L3.1-8B-I) on C Pointer Logic

I recently installed llama-cli on my Linux Debian laptop. Rather than using wrappers like lmstudio or Jan, I'm using 'native inference' - meaning that I'm running the llama-cli binaries directly on my hardware with Linux commands and flags. By compiling from source, I was able to target specific CPU flags like AVX-512 to get the most from my machine.

For this test, I used the following SLM:
Llama 3.1 8B Instruct (Q4_K_M GGUF) - these values are important and included for valid benchmarking, but for easier reading I'll refer to the model as Llama-3.1 for the rest of the article.

I wanted to see how the model would do when given the task of analyzing a C program containing pointer logic. I'm an advocate of these SLMs but we have to judge them fairly. Keep in mind that this is an 8B parameter model. By comparison, industry estimates place OpenAI's GPT-4o and Gemini 2.0 Pro between 1.5 and 2 trillion parameters, while Grok-4 has been confirmed at about 3 trillion!

Here's the C program -
bash

#include <stdio.h>
#include <stdlib.h>
#include <string.h>

int main() {
    char *buffer = malloc(100 * sizeof(char));

    if (buffer == NULL) {
        return 1;
    }

    strcpy(buffer, "Llama 3.1 is running close to the metal.");
    printf("Buffer content: %s\n", buffer);

    // Intentional bug: No free(buffer) here.
    return 0;
}

The model was given the code and then this prompt:

Explain the memory allocation here step-by-step.

Llama-3.1's response:

Here's a step-by-step explanation of the memory allocation in the code:

1. char *buffer = malloc(100 * sizeof(char));:

   • malloc is a function in C that dynamically allocates a block of memory of a specified size.
   • 100 * sizeof(char) is the size of the memory block to be allocated in bytes. Since sizeof(char) is 1 by default, this is equivalent to simply 100.
   • The allocated memory block is stored in the variable buffer, which is a pointer to the first character in the block.
   • The malloc function returns a null pointer if it fails to allocate the requested amount of memory.

2. if (buffer == NULL) { ... }:

   • This is a check to see if the memory allocation was successful. If malloc fails, buffer will be a null pointer.
   • If buffer is a null pointer, the program exits with an error code of 1.

3. strcpy(buffer, "Llama 3.1 is running close to the metal.");:

   • strcpy is a function that copies a string from one location to another. In this case, it copies the string "Llama 3.1 is running close to the metal." into the allocated memory block pointed to by buffer.
   • Since the allocated memory block is 100 bytes in size, this string will be copied successfully, but the remaining 86 bytes (100 - 14) will be left uninitialized.

4. free(buffer);:

   • free is a function that deallocates a block of memory that was previously allocated with malloc.
   • This line is necessary to prevent a memory leak. When malloc allocates memory, it does not automatically deallocate it when the program exits, so it's up to the programmer to free the memory when it's no longer needed.

Note: In a real-world program, you would typically want to check the return value of malloc to ensure that the memory allocation was successful. However, in this example, the check is omitted for simplicity.

End of Llama-3.1's response:

I'm an engineering tech, not a C programmer. I've spent some years learning C - an ongoing process. But I'm in no way qualified to understand, much less debug even this level of C program, nor the model's response. I deferred to Gemini for this task:

---

Gemini - analysis of Llama-3.1's test results

Llama's response is a solid "Solver" output, but it actually contains a classic LLM "hallucination" regarding this specific code.

The Catch
Llama claims in Step 4 that the code includes free(buffer);. Look back at the source: it doesn't.
The model "hallucinated" the fix because it's so common in correct C code. This is a perfect example of why the 8B model needs a "Judge" (a skilled C developer) to verify its work.

Technical Breakdown
• Accuracy: It correctly explained malloc and the null check.
• The Error: It failed to notice the missing free() in the actual text, instead describing what should be there.
• Math Check: It calculated 86 remaining bytes. "Llama 3.1 is running close to the metal." is actually 39 characters plus a null terminator (40 bytes). 100 - 40 = 60. It missed the math.

This first test proves why you should always double-check an 8B model's "Solver" logic on code — it often sees what it expects to see rather than what is actually on the screen.

End Gemini's Response

---

Gemini's final paragraph echoes what so many of us are beginning to accept - if you're going to use AI to write, test and debug code, you need to have a skilled human (judge) included in the process.

Despite the model's errors, I believe that this test still confirms what I'm trying to convey with these articles. That these SLMS are NOT toys - they're very, very capable within constraints - they require precise prompting and an acceptance of their limitations.

As a hardware guy, I view it as a success - that I was able to run an 8B model close to the metal on my Linux machine - and did NOT use my GPU at all. My PC having AVX-512 for the vector math, definitely helps a lot. But I ran a smaller model on my Dell Optiplex desktop using only AVX-2 with good results.

For me this is a win - I'm not focusing on the flaws in these small language models nor their limitations - I see them as well worth exploring for all of us who want to run AI on our own silicon. It just takes work on our part to work with them in the right way. I plan to see which models work on which hardware and what each IS able to do. There are no negatives here.

Ben Santora - January 2026