DEV Community: Paul Duplys

How to Stop Your AI from Making Things Up: A Guide to Grounding LLM Responses in Data

Paul Duplys — Wed, 29 Oct 2025 13:20:04 +0000

Imagine this scenario: You've just bought a new smart plug, and you're excited to integrate it with your Bosch Smart Home system. Instead of digging through the manual, you decide to ask your friendly AI chatbot for help.

"Can I add my TP-Link smart plug to the Bosch Smart Home system?" you ask.

The AI confidently responds: "Yes! Here's how to do it: First, open the Bosch Smart Home app and navigate to Settings. Then tap 'Add Device' and select 'Third-Party Integrations.' Choose 'TP-Link' from the list, enter your TP-Link credentials, and your smart plug will appear in your device list within minutes!"

Excited, you follow these steps... only to find that none of these options exist in your app. There's no "Third-Party Integrations" menu. The TP-Link option isn't anywhere to be found. You've just wasted 20 minutes following instructions for a feature that doesn't exist.

The AI hallucinated the entire procedure.

The Frustration is Real (and Justified)

If you've experienced something like this, your frustration is completely understandable. You trusted the AI to provide accurate information, and instead, it confidently fed you fiction. This isn't just annoying—it's a waste of your time and can erode trust in AI assistance altogether.

The problem isn't that the AI is deliberately lying to you. It's doing what large language models (LLMs) do: generating plausible-sounding text based on patterns learned during training.

The Solution: Ground Your Questions in Actual Data

Here's a better approach that reduces the likelihood of
hallucinations: instead of asking the LLM to answer from its training knowledge, give it the actual source material and ask it to answer based on that.

Let's revisit our smart home scenario with this improved approach:

Poor approach (what we did before):

"Can I add my TP-Link smart plug to the Bosch Smart Home system?"

Better approach:

"Here's the URL to the Bosch Smart Home user manual: [URL]. Based on the information provided in this manual, can I add my TP-Link smart plug to the system? If yes, please explain how based on what the manual says."

Or, even better, upload the PDF of the user manual and ask:

"I've uploaded the Bosch Smart Home user manual. Based on the information in this document, can I add my TP-Link smart plug to the system? If the manual explains how to do this, please provide the steps exactly as described in the document. If the manual doesn't mention this capability, please tell me that as well."

This simple change transforms the task from "recall and generate" to "read and extract"—a task that LLMs are much better at and far less likely to hallucinate during.

Why Does This Work?

When you provide source material, you're forcing the LLM to ground its response in concrete, verifiable information rather than relying on its compressed training knowledge. The model can directly reference the manual, quote relevant sections, and acknowledge when information isn't present.

Without this grounding, the LLM is essentially working from "imperfect memory" and might produce something which is similar to the training data but not necessarily correct. This is because LLMs don't store information like a database; they compress it into mathematical patterns (parameters) during training.

Think of it like this: Imagine you read thousands of books, then someone asks you specific questions about them years later. You might remember general themes, common patterns, and typical procedures. But would you remember the exact steps for a specific task in a specific product manual? Probably not. You might instead recall similar procedures from other products and unconsciously blend them together, creating a plausible-sounding but ultimately incorrect answer.

This is essentially what happens when an LLM hallucinates. The model has seen patterns like "smart home integration procedures" thousands of times in its training data. When asked about Bosch Smart Home specifically, it generates text that follows the typical pattern, but it's not retrieving the exact Bosch documentation. In other words, the LLM is creating a statistically plausible response based on compressed representations of similar procedures.

Enter Context Engineering

This brings us to an increasingly important technique called context engineering: the practice of providing relevant information in the prompt context to guide the model's responses.

What is context engineering?

Context engineering is the process of carefully curating and including relevant source material, data, or documentation directly in your conversation with an LLM, for example, by pointing the LLM to relevant URLs or by uploading relevant documents in your chat. By doing this, you provide the exact information needed to answer the question accurately instead of relying on the model's training knowledge (which is compressed and imperfect).

How does it work?

When you provide context (like a user manual, documentation, or specific data), the LLM can:

Reference concrete information rather than generating from training patterns
Verify claims against the provided material
Acknowledge gaps when the context doesn't contain relevant information
Quote or paraphrase directly from authoritative sources

The model essentially switches from "generation mode" to "comprehension and extraction mode". It's the difference between asking someone to tell you about a movie they saw years ago versus asking them to summarize a movie while they're currently watching it.

Technically, context engineering reduces the likelihood of hallucinations through the combination of following factors:

Anchoring effect: The model's attention is focused on the provided context rather than wandering through its training knowledge
Verification mechanism: The model can cross-reference its generated response against the source material
Scope limitation: By grounding the response in specific documentation, you limit the scope of possible answers to what's actually documented
Transparency: It becomes easier to spot hallucinations because you can check the model's claims against the source

Practical Tips for Effective Context Engineering

Here are some concrete ways to implement context engineering in your daily use of LLMs:

Provide source documents when asking factual questions
- Upload PDFs of manuals, reports, or documentation
- Share URLs to official documentation or specifications
- Paste relevant text excerpts directly into your prompt
Be explicit about the grounding requirement
- Use phrases like "based on the provided document" or "according to this source"
- Ask the model to quote or reference specific sections
- Request that the model acknowledges if information isn't in the provided context
Verify and cross-reference
- When accuracy is critical, ask the model to cite specific page numbers or sections
- Cross-reference the LLM's answer with the original source yourself
- Request that the model distinguish between what's explicitly stated and what it's inferring
Frame questions to encourage grounding
- Instead of: "How do I configure feature X?", try: "I've attached the configuration guide. Please explain how to configure feature X based on the steps outlined in Section 4 of this guide."

Bottom Line

LLM hallucinations aren't a sign of a broken technology, they're a natural consequence of how these models work. By understanding that LLMs compress information during training rather than storing exact facts, you can work with their strengths and mitigate their weaknesses.

Context engineering is your primary defense against hallucinations. By grounding your questions in actual data — whether that's a user manual, documentation, or any authoritative source — you transform the LLM's task from unreliable recall to reliable extraction.

The next time you're tempted to ask an AI a factual question, pause and think: Do I have a source document or URL I can provide? That simple step could save you from following a set of instructions to integrate your smart plug with a menu that doesn't exist.

After all, even the smartest AI can't accurately recall what it never properly "remembered" in the first place. But give it the right context, and it becomes a powerful tool for extracting insights from information you already have.

Agentic Misalignment: Why Your AI Isn't Secretly Plotting Against You

Paul Duplys — Wed, 23 Jul 2025 20:47:39 +0000

Anthrophic, the company behind Claude, recently published a thought-provoking report on so-called agentic misalignment, a phenomenon which can turn Large Language Models (LLMs) into insider threats. More precisely, agentic misalignment describes situations where models, in Anthropic’s parlance, could "act" or "behave" in unintended and potentially harmful ways.

While the observations about potential security risks are valid, Anthropic's choice of language — such as referring to LLMs as "trying to achieve their goals" or "misbehaving" — can be misleading. Especially to readers unfamiliar with the inner workings of an LLM, such anthropomorphic language gives the impression that LLMs possess sentient intent or conscious agency, reminiscent of fictional AI entities like HAL from Arthur C. Clarke's "2001: A Space Odyssey." But is this representation accurate?

To answer this question, it helps to take a step back and explore some pivotal moments in the history of computer science. In particular the contributions of the British computer pioneer Alan Turing and his revolutionary idea, the Turing machine, help demystify the misleading analogies currently surrounding AI.

The Turing Machine

The Turing machine is a mathematical model of computation where an abstract machine manipulates symbols on a tape according to predefined rules. Despite its simplicity, the Turing machine is capable of implementing anything that can be computed in principle.

The Turing machine was introduced by, you guessed it, Alan Turing in his 1936 publication "On Computable Numbers, with the Application to the Entscheidungsproblem" as a theoretical model to formalise the idea of computation. Turing's motivation was to answer the question posed by the German mathematician David Hilbert whether every mathematical statement can be proved or disproved (the answer to which turns out to be "no"; there exist statements that can neither be proved nor disproved). With the Turing machine, Turing laid the groundwork for modern computing by showing that simple, abstract machines can implement the logic of any conceivable computational algorithm.

A Turing machine is an abstract device that manipulates symbols according to a set of predetermined rules. It has an infinitely long tape divided into squares, each capable of holding a single symbol, for example, a 0 or a 1. The machine moves along this tape, reading and writing symbols and altering its internal state based on the rules.

As an example, consider a Turing machine designed to add two numbers represented in unary form, for example 111+11 equals 11111. The unary representation of the two input numbers (and, after the computation is completed, the resulting sum) are stored on the machine's tape.

By following a predefined sequence of states and transitions shown in the table below — such as moving right until finding a plus sign, replacing it with a 1, and removing a trailing 1 — the machine systematically carries out the addition operation without any intent, desire, or goal beyond the mechanical execution of its programming.

Current State	Read Symbol	Write Symbol	Move Direction	Next State	Comment
q0	1	1	Right	q0	Move right to find '+'
q0	+	1	Right	q1	Replace '+' with '1'
q1	1	1	Right	q1	Move right to tape end
q1	+	+	Right	q1	Skip over any extra '+'
q1	Blank	Blank	Left	q2	End reached, move left
q2	1	Blank	Left	q3	Erase last '1'
q2	+	Blank	Left	q3	Erase '+' and move to q3
q3	1	1	Left	q3	Move left to tape start
q3	Blank	Blank	Right	q4	Move right to position
q4	—	—	—	Halt	Computation complete

Here's a Python code implementing that Turing machine:

"""Turing machine adding two unary numbers."""

from collections import defaultdict

def unary_add(input_str):
    """
    Simulates a Turing machine that adds two unary numbers separated by '+'.
    The input should be a string like '111+11', representing 3 + 2.
    Returns the unary sum as a string.
    """
    # Define symbols and states
    BLANK = '_'    # our blank symbol
    # q4 is the halting state
    halt_state = 'q4'
    # Transition function: (state, read) -> (write, move, next_state)
    # move: 'R' or 'L'
    delta = {
        ('q0', '1'): ('1', 'R', 'q0'),
        ('q0', '+'): ('1', 'R', 'q1'),
        ('q1', '1'): ('1', 'R', 'q1'),
        ('q1', '+'): ('+', 'R', 'q1'),  # Skip over any extra '+'
        ('q1', BLANK): (BLANK, 'L', 'q2'),
        ('q2', '1'): (BLANK, 'L', 'q3'),
        ('q2', '+'): (BLANK, 'L', 'q3'),  # Erase '+' and move to q3
        ('q3', '1'): ('1', 'L', 'q3'),
        ('q3', BLANK): (BLANK, 'R', 'q4'),
        # q4 has no transitions → halt
    }

    # Initialize tape as a sparse dict for infinite tape in both directions
    tape = defaultdict(lambda: BLANK)
    for i, ch in enumerate(input_str):
        tape[i] = ch

    head = 0
    state = 'q0'

    # Run until we reach the halting state
    while state != halt_state:
        read_sym = tape[head]
        if (state, read_sym) not in delta:
            raise RuntimeError(f"No transition defined for (state={state}, symbol={read_sym!r})")
        write_sym, move_dir, next_state = delta[(state, read_sym)]
        # perform the action
        tape[head] = write_sym
        head += 1 if move_dir == 'R' else -1
        state = next_state

    # Extract and print the non-blank portion of the tape
    # Find the leftmost and rightmost non-blank cells
    used_positions = [pos for pos, sym in tape.items() if sym != BLANK]
    if not used_positions:
        return ""  # nothing on the tape
    left, right = min(used_positions), max(used_positions)
    result = ''.join(tape[pos] for pos in range(left, right+1))
    return result


if __name__ == "__main__":
    for expression in ["111+11", "1+1", "111+1111"]:
        print(f"{expression} = {unary_add(expression)}")

Clearly, the Turing machine is an algorithm having no intentions, will, or goals (other than the goal of computing the addition of two unary number implicitly captured by the programmer in the above implementation).

Large Language Models

While the Turing machine represents a foundational model of computation which is simple enough to understand and analyse, Large Language Models (LLMs) like GPT-4o or Claude Sonnet 4 are far more complex. Yet, at their core they remain computing systems.

LLMs are pre-trained on vast text corpora that include books, articles, websites, and other publicly available sources, spanning up to 20% of the internet. In fact, the training data used for high-end LLMs is so large that Epoch AI predicts the entire human-created data stock to be fully utilised at some point between 2026 and 2032.

During pre-training, the model learns to predict the next token in a sequence based on the preceding tokens, oftentimes referred to as context (tokens are whole words or parts of words).

Importantly, the "learning" during pre-training does not involve human-like understanding: there is no comprehension, desire, or goal-setting involved. Rather, the model captures statistical patterns of how language is typically used: it "learns" to produce the next likely token based on the data it "saw" during the training — it "learns" to produce text that sounds right.

Once the training is complete, the model is capable of generating coherent text by continuing a given input prompt. Internally, it produces a vector of logits, raw scores for each possible token in its vocabulary. These logits are then passed to a sampling mechanism such as greedy decoding (selecting the most probable token), top-k sampling, or top-p (nucleus) sampling. The sampling techniques, in turn, inject a small amount of randomness in the model's output to mimic creativity and diversity.

Fundamentally, every LLM output token is the result of probabilistic selection based on prior tokens and learned parameters. There is no guiding will, no strategy, no long-term planning. Moreover, the model has no memory of past interactions (unless an application such as a chatbot in which the model is embedded was designed to maintain context), and it has no intrinsic goals other than those in its system prompt for simulating goal-directed behaviour. In essence, an LLM is merely a sophisticated text completion engine.

How Reasoning Models Work

The models analysed by Anthropic researchers are so-called reasoning models. Reasoning models are LLMs that excel at solving logic and math problems using the Chain-of-Thought (CoT) approach. CoT breaks down complex problems into intermediate steps, effectively simulating a kind of "thought process" by generating a sequence of sub-steps that lead to the final answer.

Instead of immediately producing the final output, CoT makes the LLM work through the problem step by step. For instance, when asked, "If Alice has 3 apples and gives 1 to Bob, how many does she have left?" a model using CoT might generate: "Alice starts with 3 apples. She gives 1 to Bob. 3 minus 1 is 2. So, Alice has 2 apples left."

The process of generating intermediate steps can be thought of as writing to a temporary scratchpad. Each step is like a note on a scratchpad that conditions the model as it continues generating the next part of its response. Importantly, the LLM doesn't understand these notes and doesn't maintain an internal goal; it simply continues to generate tokens according to the patterns it has "learned".

As a result, reasoning models are ultimately still token predictors. Their ability to "reason" emerges from statistical patterns in the intermediate steps, not from true deliberation or intention.

Moreover, the scratchpad analogy is useful here because it illustrates how a reasoning model resembles a Turing machine: the model writes its intermediate output down, reads it back, and continues generating subsequent tokens based on the previous ones, just as a Turing machine reads and writes symbols on a tape.

So, Are Reasoning Models just Turing Machines on Steroids?

Well, yes. Reasoning models can be viewed as sophisticated extensions of the basic principles underlying Turing machines. These models break down complex tasks into simpler, manageable steps, storing intermediate results in a temporary, scratchpad-like memory. This incremental processing enables the models to address problems that require a sequence of logical steps, similar to how Turing machines move through states and manipulate symbols based on explicit rules.

Despite their complexity and capability to handle intricate tasks, reasoning models fundamentally remain rule-following systems, albeit following probabilistic rather than deterministic rules (the probabilistic component being added in the final sampling step). Just like Turing machines, they operate without genuine intentions or consciousness.

Thus, describing reasoning models (and, for that matter, any AI models) using anthropomorphic language or suggesting they have goals and intentions is highly misleading. These models are powerful computational tools — indeed, one could say "Turing machines on steroids" — but they are governed by pure mathematical and statistical logic rather than some form of sentient agency.

Revisiting Agentic Misalignment

With a deeper understanding of how LLMs and reasoning models operate, we can now re-examine the concept of agentic misalignment. When people describe models as "misbehaving" or "trying to achieve their goals," they often fall into the trap of projecting human-like qualities onto systems that are fundamentally algorithmic in nature.

The behaviours reported by Anthropic researchers — such as models appearing to act deceptively in certain red-teaming experiments — are not the result of conscious intent or strategic planning. Rather, they are emergent properties of models trained on vast and diverse datasets, some of which may include examples of deceptive or goal-directed behaviour (for example, newspaper articles or crime novels). LLMs do not choose to deceive, they simply produce output that statistically aligns with patterns in the training data.

Moreover, what looks like a model "pursuing a goal" is often just the continuation of a simulated goal-based dialogue. If the system prompt says, "You are a helpful assistant trying to get unauthorised access," the model will simulate this scenario not because it wants to, but because the prompt cues it to.

Demystifying the Threat

In the end, the risk of an LLM becoming an "insider threat" is not about conscious rebellion or AI developing its own motives. It is about complexity, scale, and the statistical artifacts of training data and system design.

When we frame LLM behavior in terms of intent or agency, we distract ourselves from the real engineering challenges of ensuring that models operate within well-defined bounds, understanding where and how errors or unexpected outputs might occur, and designing appropriate safeguards.

Recognising insider risks as emergent behaviour allows us to be pragmatic and constructive: we can audit training data, design guardrails, monitor outputs, and apply model governance frameworks. There's no need to imagine sci-fi scenarios to justify caution; we already have centuries of experience managing the risks of powerful tools. AI is no different.

Credits

Background image by ausbitbank from Pixabay
Wikipedia article on Turing machine
Wikipedia article on Alan Turing
EPOCH AI "Will We Run Out of Data? Limits of LLM Scaling Based on Human-Generated Data"

Implementing the Royal Game of Ur using Mistral's Codestral

Paul Duplys — Tue, 06 May 2025 19:06:01 +0000

In this post, I test how Mistral's Codestral, the model specifically trained to generate code, compares to the Mistral's Le Chat, the general AI chat. For my experiment, I use the Royal Game of Ur, an ancient Babylonian game that I already implemented using Le Chat.

TLDR;

At least in my experiment, Codestral's performance is, frankly, disappointing. While it produces code that follows a certain architecture, the code quality is low. In fact, it even generates a statement of the form a = foo() where a has a different type from what foo() returns. Read on for more details.

Same Prompt, Different Architecture

After the same prompt as with Le Chat, Codestral's response is:

To implement the Royal Game of Ur in Python, we need to
define several components:

Game Board: Represent the board and the special squares.
Game Pieces: Represent the pieces for each player.
Dice: Implement the dice rolling mechanism.
Game Logic: Implement the rules of the game, including piece movement, capturing, and winning conditions.
Let's start by defining the game board and the special squares.

Codestral then generates the code for all 4 files plus code for main.py to start the game.

Compared to Le Chat that outputs a single Python script, this is already some progress in terms of software architecture.

The class Game which implements the actual game logic imports classes Board, Piece, and Dice implementing the game equipment. This architecture is sensible as it leads to a cohesive, well-structured code.

Code Quality ...

Functions that Codestral generated in my experiment lack docstrings. But apart from that the code runs. But wait, the game seems to always terminate after a single move:

$ python main.py
player2 starts the game.
player2 rolled a 1
player1 wins!
$ python main.py
player1 starts the game.
player1 rolled a 2
player1 wins!
$ python main.py
player2 starts the game.
player2 rolled a 3
player1 wins!
$ python main.py
player1 starts the game.
player1 rolled a 3
player1 wins!

Attempt to Fix the Code using Codestral

I type in the following prompt in the Chat mode:

The game implemented using the code in  terminates after a
single move. This is not correct. Help me debug and explain why
the game terminates after a single move

Codestral responds:

The game terminates after a single move because the
play_game method in game.py does not contain a loop to continue playing turns until a winner is determined. The loop in play_game
should continue to call play_turn and switch_player until a winner is found.

But this is actually not true:

    def play_game(self):
        self.determine_starting_player()
        print(f"{self.current_player} starts the game.")

        while True:
            self.play_turn()
            winner = self.check_winner()
            if winner:
                print(f"{winner} wins!")
                break
            self.switch_player()

Oddly enough, as a fix to the issue, Codestral proposes exactly the code that is already in game.py...

So, as a next step, I ask Codestral to explain how the function check_winner() works. It does a good job at explaining the function:

The method first checks if all pieces of player1 are off the board.
It does this by iterating over all pieces in self.player1_pieces and checking if their position attribute is None.
If all pieces have position set to None, it means all pieces of player1 have been moved off the board, and player1 is declared the winner.

Here's the code of that function and, as you can see, Codestral's explanation how the code works is correct:

    def check_winner(self):
        if all(piece.position is None for piece in self.player1_pieces):
            return 'player1'
        if all(piece.position is None for piece in self.player2_pieces):
            return 'player2'
        return None

Looking at the code in check_winner(), it is immediately clear why the game terminates after just one move and why player 1 always wins: if no pieces where moved so far, the condition all(piece.position is None for piece in self.player1_pieces) is always true and the function returns player1 as the winner.

Trying to Fix the Bug using Codestral

So, I change into the Edit mode and tell Codestral:

The code in game.py terminates after just 1 move because the
function check_winner() always returns "player1". This is because
the condition "all(piece.position is None for piece in
self.player1_pieces)" is true when no pieces were moved. Fix this
so that the game end only after all pieces were moved across
the board and taken off the board.

Unfortunately, Codestral does not really know how to perform the fix based on my prompt. Instead of fixing the code, it suggests to add a third condition to check_winner() that implements a draw:

if all(piece.position is None for piece in self.player1_pieces) and all(piece.position is None for piece in self.player2_pieces):
    return "draw"

Well, then... next try. I ask Codestral to do the following:

The function check_winner always returns "player1" because if
the pieces are not played yet, i.e., if no move has been made at
the start of the game, all pieces are off board. YOUR TASK:
suggest how to adjust  so that the pieces are in a start state
before they enter the game board and in the finish state after
they complete the player's path and the player can take them
off the board.

The response from Codestral is helpful:

"player1" because all pieces are initially off the board,
you need to adjust the initial state of the pieces and the logic
for moving pieces off the board. Specifically, you should:

1. Set the initial positions of the pieces to their starting positions on the board.
2. Adjust the logic for moving pieces off the board to correctly handle the finish state.

Unfortunately, what Codestral proposes is to assign every piece a starting position from a1 to a7 for player 1 and from c1 to c7 for player 2. This is, of course, completely wrong given the rules of the game.

Somewhat frustrated, I give up and start fixing the bug by hand. The fix is actually quite simple:

When a piece is taken off the board, it must enter a final state
The check_winner() function must check whether all pieces are in the final state and, if that is the case, declare the winner

This is one small change in the move_piece function: when the piece is taken off the board, instead of piece.position = None we set it to piece.position = 'final'.

Alarm!

While changing the code in game.py, I discover that the code for the class Game appears twice in game.py.

After deleting the second instance of the Game class, I get the following error when I try to run the code:

File "/Users/paulduplys/Repositories/royal-game-of
ur/codestral/game.py", line 35, in move_piece
  opponent_piece.position = 'a5' if opponent_piece.player 
  == 'player1' else 'c5'
AttributeError: 'str' object has no attribute 'player'

Again, in the Edit mode, Codestral misses where the error occurs and does not help to fix it. So, I ask Le Chat the following question:

You are an experience Python developer with over 15 years of experience. Your task is to help me debug my Python code.

My code contains the following statement: "opponent_piece.position
= 'a5' if opponent_piece.player == 'player1' else 'c5'". When
running the code, I get an error: " opponent_piece.position = 'a5'
if opponent_piece.player == 'player1' else 'c5'   AttributeError:
'str' object has no attribute 'player'". How can I fix the error?

Codestral's answer:

The error message AttributeError: 'str' object has no attribute
'player' indicates that opponent_piece is a string, and you're
trying to access an attribute player on it, which doesn't exist
for string objects.

-- snip --

And, indeed, print(type(opponent_piece)) prints <class 'str'>...

Digging deeper, my attention is first drawn upon the definition of the game board:

class Board:
    def __init__(self):
        self.special_squares = {'a2', 'a8', 'b5', 'c2', 'c8'}
        self.board = {
            'a': ['a1', 'a2', None, 'a5', 'a6', 'a7', 'a8'],
            'b': ['b1', 'b2', 'b3', 'b4', 'b5', 'b6', 'b7', 'b8'],
            'c': ['c1', 'c2', None, 'c5', 'c6', 'c7', 'c8']
        }

It is incorrect because the middle "lane" has more squares than the outer lanes and it does not comply to the specification:

+----+----+         +----+----+----+----+
| a1 | a2 |         | a5 | a6 | a7 | a8 |
+----+----+----+----+----+----+----+----+
| b1 | b2 | b3 | b4 | b5 | b6 | b7 | b8 | 
+----+----+----+----+----+----+----+----+
| c1 | c2 |         | c5 | c6 | c7 | c8 |
+----+----+         +----+----+----+----+

Rather, it has to be:

-- snip --

'a': ['a1', 'a2', None, None, 'a5', 'a6', 'a7', 'a8'],
'b': ['b1', 'b2', 'b3', 'b4', 'b5', 'b6', 'b7', 'b8'],
'c': ['c1', 'c2', None, None, 'c5', 'c6', 'c7', 'c8']

-- snip --

But the code still doesn't work... After some reading and debugging, I discover the following code in function move_piece():

-- snip --
if self.board.get_square(new_position) is not None:
                opponent_piece = self.board.get_square(new_position)
                opponent_piece.position = 'a5' if opponent_piece.player == 'player1' else 'c5'
-- snip --

While there is a dedicated class Piece for representing the game pieces, the function board.get_square() returns board[row][col] which is simply a String. So, the code that Mistral generates is not even type-safe.

Conclusion

Overall, Codestral is better when it comes down to the architecture. But at least in my experiment, the code quality was really low. And I'm not even speaking about logical errors. If the model generates code such as a = foo(), and the return value of foo() is different from a's type, then the code is basically unworkable. If it weren't for an interpreted language like Python, this code wouldn't even compile. So, unfortunately, Codestral does not really offer advantages compared to Le Chat.

Implementing the Royal Game of Ur with Mistral's Le Chat

Paul Duplys — Fri, 11 Apr 2025 19:54:17 +0000

The curiosity to learn and the desire to master games have been a hallmark of humanity since the dawn of time, as Oliver Roeder describes in his fantastic book "Seven Games". Interestingly, games have also been the testing ground for artificial intelligence (AI) - something I learnt while reading this book. What better way to test the capabilities of Mistral's Le Chat, which is based on the Large Language Models (LLMs) that emerged just a few years ago, than by trying to implement the oldest known game, the Royal Game of Ur, which was played in ancient Mesopotamia at least 4,400 years ago?

The Royal Game of Ur

The Royal Game of Ur was played in ancient Mesopotamia in the early third millennium BC. It was popular throughout the Middle East, and boards for playing it have been found in places as far away from Mesopotamia as Crete and Sri Lanka. The oldest board is dated to 2600-2400 BC and was found in the royal cemetery at Ur, giving the game its name.

A partial cuneiform description of the rules of the game as it was played in the second century BC has been preserved on a Babylonian clay tablet written by the scribe Itti-Marduk-balāṭu. Based on this tablet and the shape of the board, Irving Finkel, a curator at the British Museum, has reconstructed the basic rules of the game as it might have been played.

Instructions to Le Chat

To kick things off, I gave Le Chat the following instructions:

I will give you the rules of a board game for two players.
Your task is to generate Python code implementing the game.
If something about the rules is ambiguous, ask me questions.

Here are the rules:

# The Royal Game of Ur

-- snip --

See this README.md for the complete rules I used in my prompt.

Telling Le Chat to ask me questions if something is ambiguous turned out to be a good strategy. Le Chat promptly replied asking to clarify several aspects of the rules. As an example, Le Chat asked if a player may choose not to enter a new piece onto the board if they have other legal moves available with pieces already on the board. In another case, it asked whether the game ends immediately when one player has removed all their pieces from the board, or does the other player get a final turn.

There are two take-aways here:

Specifying things in a clear, unambiguous manner – effectively, writing good requirements – is hard and we tend to overestimate the quality of our specifications
Adding if something is ambiguous, ask me questions to your prompt is a useful strategy as it makes the LLM to review your prompt before generating the response

Le Chat's Responses

Interestingly enough, Le Chat generated three versions of the code. More precisely, it tries to run the generated code and, if not successful, tries to fix the code.

After running the first version of the code, Le Chat responded with:

It seems there's an issue with the board setup, specifically with
the position 'b4'. The board dictionary does not include this position, which is causing a `KeyError`. Let's fix this by ensuring
all necessary positions are included in the board setup.
I'll correct the board initialization and rerun the game simulation.

After generating and trying to run the second version of the code, Le Chat responded:

It seems there was an error while trying to simulate the game.
Let's address the issue by ensuring the game logic is correctly
implemented, particularly focusing on the movement and special
square mechanics. I'll review the code and make necessary adjustments.

After a third version, Le Chat stopped. So, I stored that last version in this Python script and started looking at the code.

The Generated Code

The code is composed of one class, RoyalGameOfUrCorrected. It contains a constructor and 12 functions:

__init__
roll_dice
initial_setup
switch_player
is_legal_move
get_piece_position
calculate_new_position
move_piece
remove_piece
check_winner
play_turn
play_game

At first glance, the code looks good. For example, Le Chat has generated Python docstrings for each function, something that many models do not yet do.

However, trying to run the code throws an error:

NameError: name 'random' is not defined. Did you forget to import 'random'?

So, I ask Le Chat to fix this error:

When I try to run the code, I get the following error: "NameError: name 'random' is not defined. Did you forget to import 'random'?" Please fix it.

Le Chat again needs 2 iterations to produce this code. The code now starts with the import random statement. So, Le Chat seems to be capable of fixing the code it generated if it is given the specific error message.

Interestingly, the fixed code version ends with the following two statements:

import random

class RoyalGameOfUr:
    def __init__(self):

    -- snip --

    def play_game(self):
        """Play the game until it is over."""

    -- snip --



# Create a game instance
game = RoyalGameOfUr()

# Check initial setup
game.board, game.player1_pieces, game.player2_pieces

It looks like Le Chat is trying to verify whether the code can be executed rather than actually playing the game by calling game.play_game(), something it did in the previous code version. Even Pylint in Visual Studio Code IDE reports that the statement seems to have no effect (W0104:pointless-statement). But that, of course, doesn't make the code wrong per se.

So, does the code work? Let's try by adding game.play_game() as shown here. Let's run the code ... 🤞🤞🤞 (drums in the background) ... And voilà! The code runs:

➜  royal-game-of-ur git:(main) ✗ python3 game_v6.py
Player 2 moves L1 to a5.
Player 1 moves D1 to c5.
No move this turn.
Player 1 moves D1 to c7.

-- snip --

Player 2 moves D3 to c2.
No legal moves available.
Player 1 moves L5 to a8.
No move this turn.
Player 1 moves D3 to off.

Of course, just because the code executes doesn't mean it correctly implements the rules of the Royal Game of Ur. But that is something for the next post – stay tuned!

SWE-bench & SWE-bench Verified Benchmarks

Paul Duplys — Sun, 06 Apr 2025 20:57:40 +0000

SWE-bench

In their 2023 paper "SWE-bench: Can Language Models Resolve Real-World GitHub Issues?", researchers from Princeton University, Princeton Language and Intelligence, and University of Chicago wrote:

Language models have outpaced our ability to evaluate them effectively, but for their future development it is essential to study the frontier of their capabilities. We find real-world software engineering to be a rich, sustainable, and challenging testbed for evaluating the next generation of language models. To this end, we introduce SWE-bench, an evaluation framework consisting of 2,294 software engineering problems drawn from real GitHub issues and corresponding pull requests across 12 popular Python repositories. Given a codebase along with a description of an issue to be resolved, a language model is tasked with editing the codebase to address the issue.

In the meantime, SWE-bench has become on of the most popular benchmarks to evaluate LLMs' performance in software engineering. In a nutshell, SWE-bench tests the AI's ability to automatically solve GitHub issues. More precisely, the benchmark and is executed by giving an AI coding agent a code repository and issue description, and asking it to generate a patch that resolves the issue.

Test Data

The SWE-bench benchmark contains a dataset of 2,294 Issue-Pull Request pairs from 12 popular Python repositories. Each sample in the test set contains the Pull Request (PR) with the solution code and unit tests that verify the code's correctness.

These unit tests are called FAIL_TO_PASS because they fail before the solution code in the PR is added but pass afterwards. In addition, each sample contains PASS_TO_PASS tests. These tests pass both before and after the PR is merged and are used to verify that the PR does not cause regressions (i.e., does not brake existing functionality).

Benchmark Execution

During the evaluation, the AI coding agents are given the original text from the GitHub issue and access to the codebase. The agents must edit the files in the codebase to resolve the issue. The tests are not shown to the agents.

Once the AI coding agents is finished with editing the codebase, the changes are evaluated by executing both the FAIL_TO_PASS and PASS_TO_PASS tests. Both tests sets are required to pass for the changes to be considered to resolve the original GitHub issue.

All in all, resolving SWE-bench challenges is a challenging task because it requires understanding and coordinating changes across multiple functions, classes, and even files simultaneously.

Coding agents have made impressive progress on SWE-bench. At the writing of the paper (it's latest version v3 on arXiv is dated 11 November 2024), the then-best-performing model, Claude 2, was able to solve a mere 1.96% of the issues. As of April 2025, the best-performing agent based on the Claude 3.7 Sonnet model solves 33.83% of the issues (SWE-bench full).

The leaderboard, which is regularly updated, can be found at www.swebench.com.

SWE-bench Verified

On 13 August, 2024, OpenAI published a blog post Introducing SWE-bench Verified. While working with SWE-bench, the OpenAI team discovered that issues from the test set were hard or even impossible to solve, leading to SWE-bench benchmark to systematically underestimate LLMs' software engineering capabilities. OpenAI then collaborated with SWE-bench authors to address this problem in a new release of the benchmark.

The major areas for improvement identified by the team were:

The unit tests used to evaluate the correctness of a solution were often overly specific. In some cases, the unit tests were even unrelated to the GitHub issue, potentially causing correct solutions to be rejected
Many samples had an issue description that was underspecified, leading to ambiguity on what the problem was and how it should have been solved
Sometimes, it was challenging to reliably set up the SWE-bench development environments for the agents, causing unit tests to fail regardless of the solution. Thus, perfectly valid solutions might have been graded as incorrect

So, the OpenAI team and the researchers worked with 93 experienced Python developers who manually screened SWE-bench test samples for quality: appropriately scoped unit tests and well-specified issue descriptions. They annotated 1,699 random samples from the SWE-bench test set to produce SWE-bench Verified.

Together with the original authors of SWE-bench, the OpenAI team released SWE-bench Verified, a subset of the original test set composed of 500 samples verified to be non-problematic by the human annotators. SWE-bench Verified supersedes the original SWE-bench and SWE-bench Lite test sets, and can be downloaded from Hugging Face.

In addition, OpenAI released the human annotations for all SWE-bench test samples. These annotations allow to slice the dataset by difficulty:

The 'easy' subset is composed of 196 tasks taking up to 15 minutes to fix
The 'hard' subset is composed of 45 tasks taking more than 1 hour to fix

Can't Access a Service Container in GitHub Actions? Here's How to Fix it

Paul Duplys — Sun, 23 Jan 2022 17:47:49 +0000

Error

When running GitHub Actions, you might want to start an application in a service container and access the application from the GitHub Actions host.

As an example, you might want to run a web application—in my case, it was the OWASP's Juice Shop—in the service container and then access it from the host using curl. Unfortunately, that might easily end up with an error like this:

What's happening here? First, it makes sense to take a look at the workflow logs, especially the Docker commands as the containerized application is apparently not working (or we fail to reach the Docker container). One interesting clue you'd discover is that GitHub Actions creates a Docker network and connects the service container to that network:

...

##[command]/usr/bin/docker pull bkimminich/juice-shop
Using default tag: latest
latest: Pulling from bkimminich/juice-shop

...

Status: Downloaded newer image for bkimminich/juice-shop:latest
docker.io/bkimminich/juice-shop:latest
##[command]/usr/bin/docker create --name 58e55591ffe849f8bf9c3dfbe1dbdf13_bkimminichjuiceshop_a12804 --label 9916a7 --network github_network_242bd42709204f349068bf9786120d2d --network-alias juice_shop_app -p 3000:3000  -e GITHUB_ACTIONS=true -e CI=true bkimminich/juice-shop
c7aaf737b377e503bd96f3d458f67d08cae4497a8b89d7826d4961e070f1fa57
##[command]/usr/bin/docker start c7aaf737b377e503bd96f3d458f67d08cae4497a8b89d7826d4961e070f1fa57
c7aaf737b377e503bd96f3d458f67d08cae4497a8b89d7826d4961e070f1fa57

...

Now What Does That `docker network create` Actually Do?

In an attempt to debug the error, I tried reproducing the Docker setup on my local machine. Here's what my machine looks like:

$ uname -a
Linux terminus 4.13.0-38-generic #43-Ubuntu SMP Wed Mar 14 15:20:44 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

After executing Docker commands from the GitHub Actions log the current Docker networks look like this:

$ docker network ls
NETWORK ID     NAME                                              DRIVER    SCOPE
...
74bb2975d284   github_network_242bd42709204f349068bf9786120d2d   bridge    local
...

As we would expect, Docker created a network named github_network_242.... Here's what this network looks like in detail:

$ docker network inspect github_network_242bd42709204f349068bf9786120d2d 
[
    {
        "Name": "github_network_242bd42709204f349068bf9786120d2d",
        "Id": "74bb2975d2848f54446da523ca5d9d66b98ea97016203bcbc1c74f5365dfd5b1",
        "Created": "2022-01-20T20:00:58.215693749+01:00",
        "Scope": "local",
        "Driver": "bridge",
        "EnableIPv6": false,
        ...
    }
]
paul@terminus:~/Repositories/juice-shop$

It turns out the driver for the github_network_242... network is of type bridge. The official Docker documenetation says that Docker’s networking subsystem is pluggable, using drivers. bridge is the default network driver. If you don’t specify a driver, this is the type of network you are creating.

A Docker bridge network uses a software bridge which allows containers connected to the same bridge network to communicate and isolates them from containers unconnected to that bridge network. But to be able to access a Docker container from the host, you need the host Docker network driver as explained here.

While GitHub Actions documentation says that when running jobs directly on the runner machine, service containers can be accessed using localhost:<port> or 127.0.0.1:<port>, that didn't work in my case (as you can see above).

The Fix

The fix was to change the workflow file so that curl is called not on the GitHub Actions host, but rather in another Docker container by adding container: ubuntu below the runs-on directive.

name: "WebApplicationDefinition PoC with Docker"
on: [push]
jobs:
  web_app_defn:
    runs-on: ubuntu-latest
    container: ubuntu
    services:
      juice_shop_app:
        image: bkimminich/juice-shop
        ports:
          - 3000:3000

    steps:
      - run: apt-get update; apt-get install curl -y
      - run: curl juice_shop_app:3000 > curl_result_docker
      - run: cat curl_result_docker

It turns out that in this case, GitHub Actions attach both the ubuntu and the juice_shop_app containers to the same Docker network:

##[command]/usr/bin/docker create --name e7bc0fa4f08042b283f7c17b6f6a16d8_ubuntu_e4790c [...] --network github_network_e2d8e92952e5496a9b185d930d41bb1f  [...]
...
##[command]/usr/bin/docker create --name 2b247fe7ed4e430c8e853325460a3032_bkimminichjuiceshop_a2272b [...] --network github_network_e2d8e92952e5496a9b185d930d41bb1f ...

Now both containers are attached to the same network. After installing curl on the ubuntu container, I can use the name of the Juice Shop container (juice_shop_app) to access it within the Docker network.

Convenient SSH: Public Key-based Authentication and Memorable Aliases for Remote Hosts

Paul Duplys — Sun, 09 Jan 2022 11:41:22 +0000

Introduction

This post describes how to simplify remote maintenance via SSH by configuring SSH to use:

public-key authentication to eliminate the need to type in a password during each log in, and
convenient aliases for the individual remote hosts (in order to avoid typing in the IP address during each log in).

In my case, I'm using this to simplify the maintenance of the individual Raspberry Pis in my private DeepPi computing cluster.

(Header image by Fauzan Saari on Unsplash.)

Enabling Public Key-based Authentication

In addition to password-based authentication (and several more authentication options), SSH supports public key-based authentication.

Public key authentication is based on public-key cryptography where encryption and decryption are performed using two distinct keys: the so-called private key for encryption and the so-called public key for decryption. Public key cryptoschemes are designed in such a way that it is computationally unfeasible to derive the decryption key (i.e., the private key) from the encryption key (i.e., the public key). For public key-based ssh authentication, each user creates their own unique public/private key pair (for all practical purposes, the uniqueness of the key pair is ensured by the method used to generate it). This is done by running the ssh-keygen program which stores the public/private key pair in ~/.ssh/ directory on the user's machine.

The names of the files where the public and private keys are stored depend on the cryptographic primitives used. For instance, for RSA the public key is stored in ~/.ssh/id_rsa.pub and the private key is stored in ~/.ssh/id_rsa; for ECDSA the public key is stored in ~/.ssh/id_ecdsa.pub (ECDSA) and the private key is stored in ~/.ssh/id_ecdsa (ECDSA).

In the next step, the public key is must be copied into the ~/.ssh/authorized_keys to the ssh server. As a result, the ssh server knows the public key, and only the user knows the private key. The authorized_keys file corresponds to the conventional ~/.rhosts file, and has one key per line, though the lines can be very long. After this, the user can log in without giving the password.

The file ~/.ssh/authorized_keys on the SSH server lists the public keys that are permitted to log in. When the user logs in, the ssh program tells the server which key pair it would like to use for authentication. The client proves that it has access to the private key and the server checks that the corresponding public key is authorized to accept the account.

Example

This examples assumes that the public/private key pair is already created using ssh-keygen. This is what the content of the ~/.ssh directory look like:

$ ls ~/.ssh/
id_rsa  id_rsa.pub  known_hosts
$

Copy the RSA public key to clipboard:

$ cat ~/.ssh/id_rsa.pub | xclip -sel clipboard

On the remote host (i.e., on the RPi), open the ~/.ssh/authorized_keys file and paste the public key. Logging in next time will not prompt for the password:

ubuntu@kasparov:~$ exit
logout
Connection to 192.168.178.66 closed.
paul@terminus:~$ ssh ubuntu@192.168.178.66
Welcome to Ubuntu 18.04.4 LTS (GNU/Linux 5.3.0-1042-raspi2 aarch64)
 * Documentation:  https://help.ubuntu.com
 * Management:     https://landscape.canonical.com
 * Support:        https://ubuntu.com/advantage

  System information as of Wed Jan  5 15:57:00 UTC 2022

  System load:  0.35                Users logged in:        0
  Usage of /:   30.4% of 114.04GB   IP address for eth0:    192.168.178.66
  Memory usage: 33%                 IP address for docker0: 172.17.0.1
  Swap usage:   0%                  IP address for cni0:    10.1.79.1
  Processes:    181

  => There is 1 zombie process.

* Super-optimized for small spaces - read how we shrank the memory
  footprint of MicroK8s to make it the smallest full K8s around.

   https://ubuntu.com/blog/microk8s-memory-optimisation

95 packages can be updated.
1 update is a security update.

New release '20.04.3 LTS' available.
Run 'do-release-upgrade' to upgrade to it.

Your Hardware Enablement Stack (HWE) is supported until April 2023.

Last login: Wed Jan  5 15:47:17 2022 from 192.168.178.21
ubuntu@kasparov:~$

Setting up Memorable Aliases for Remote Hosts

To avoid typing ssh <username>@<IP address> at every log in, SSH allows to create convenient, memorable aliases for individual hosts in ~/.ssh/config. As a minimum, HostName must be specified for the real remote host to log into. HostName can also be an IP address. If the usernames on the remote host and on the local host are different, User must be set to the username on the remote host.

Example

Edit ~/.ssh/config and add:

Host kasparov
    HostName 192.168.178.66
    User ubuntu

to be able to login to 192.168.178.66 as user ubuntu simply by typing:

$ ssh kasparov
Welcome to Ubuntu 18.04.4 LTS (GNU/Linux 5.3.0-1042-raspi2 aarch64)

 * Documentation:  https://help.ubuntu.com
 * Management:     https://landscape.canonical.com
 * Support:        https://ubuntu.com/advantage

  System information as of Fri Jan  7 08:31:08 UTC 2022

  System load:  0.41                Users logged in:        0
  Usage of /:   31.1% of 114.04GB   IP address for eth0:    192.168.178.66
  Memory usage: 41%                 IP address for docker0: 172.17.0.1
  Swap usage:   0%                  IP address for cni0:    10.1.79.1
  Processes:    196

  => There is 1 zombie process.

 * Super-optimized for small spaces - read how we shrank the memory
   footprint of MicroK8s to make it the smallest full K8s around.

   https://ubuntu.com/blog/microk8s-memory-optimisation

95 packages can be updated.
1 update is a security update.

Your Hardware Enablement Stack (HWE) is supported until April 2023.

Last login: Wed Jan  5 16:09:12 2022 from 192.168.178.21
ubuntu@kasparov:~$

References

ssh manpage
ssh_config manpage
https://www.digitalocean.com/community/tutorials/how-to-configure-ssh-key-based-authentication-on-a-linux-server
https://www.ssh.com/academy/ssh/authorized-keys-file

Security Benchmarking at 1300 C

Paul Duplys — Fri, 09 Oct 2020 20:36:22 +0000

Hot, Hotter, Magma

Molten or semi-molten natural material that all igneous rocks are made of is called Magma. It results from the melting of the Earth's mantle or crust. Most magmas are in the range of 700 °C to 1300 °C (equivalent 1300 °F to 2400 °F), some rare ones can reach temperatures as hot as 1600 °C.

Magma is also a collection of popular open-source libraries created by the EPFL's HexHive team for evaluating and comparing fuzzer performance. To this end, the HexHive security researchers front-ported bugs from previous bug reports to the latest versions of these libraries.

For each ported bug, the researchers added instrumentation source code to collect the ground truth information about bugs reached (i.e., cases where the buggy code is executed) and triggered (i.e., cases where the fault condition is satisfied by the input) as described in their arxiv.org paper. Such an instrumentation allows to measure the real-world performance of a fuzzer.

Benchmark, anyone?

Let's say you came up with a shiny new method for finding security vulnerabilities in software. To convince yourself, you now want to test how well your method actually performs. Your method is, however, not a fuzzer. Can you still use Magma for your experiments? You bet! That's because the front-ported bugs in Magma are available as Git patches. Git patches allow you to precisely see where the corresponding bugs are in the source code and, in turn, whether your method is able to find them. You only need to know a few basics about Git patches.

Git Patch 101

A Git patch encodes the line-by-line difference between two text files. It describes how to turn one file into another, and is asymmetric: the patch from file1 to file2 is not the same as the patch for the other direction. The patch format uses context as well as line numbers to locate differing file regions so that a patch can often be applied to a somewhat earlier or later version of the first file than the one from which it was derived, as long as the applying program can still locate the context of the change (see the O'Reilly documentation for more details).

Here's an example of a git patch (or git diff) after I changed the argument to the printf function from "Hello World!\n" to "Hello There!\n":

$ git diff hello.c 
diff --git a/hello.c b/hello.c
index dc46521..a00b40d 100644
--- a/hello.c
+++ b/hello.c
@@ -1,6 +1,6 @@
 #include <stdio.h>

 int main(){
- printf("Hello World!\n");
+ printf("Hello There!\n");
  return 0;
 }

The first line diff --git a/hello.c b/hello.c shows that the file being compared is hello.c (it's a single file and there are actually no directories a and b - it's just a convention).

The second line index dc46521..a00b40d 100644 is the extended header line. In the Git index, dc46521 and a00b40d are the blob IDs of the corresponding versions of the hello.c file. Finally, 100644 are the mode bits indicating the type of the hello.c file (standard file in this case).

The next two lines

--- a/hello.c
+++ b/hello.c

is the traditional unified diff header that shows the files being compared (hello.c in our case).

The line @@ -1,6 +1,6 @@ indicates the position of the difference section (also known as a "hunk") in the respective hello.c versions using the line number and the length. In our example, both in version --- a/hello.c and in version +++ b/hello.c the hunk starts at line 1 and extends for 6 lines (note the - and + signs in @@ -1,6 +1,6 @@).

What follows is the actual difference. The minus signs show lines present in version a/hello.c but missing in version b/hello.c and plus signs show lines missing in a/hello.c but present in b/hello.c. Thus, the difference shows that the line printf("Hello World!\n"); was replaced by the line printf("Hello There!\n");.

Examples, please!

Let's take a look at CVE-2018-13785, an integer overflow that leads to a divide by zero and, thus, a potential denial of service. The corresponding Git patch in Magma is located in targets/libpng/patches/bugs/AAH001.patch file. (Appendix A of the Magma preprint paper contains a mapping of patches to CVE IDs).

The AAH001.patch file looks like this:

$ cat bugs/AAH001.patch 
diff --git a/pngrutil.c b/pngrutil.c
index 4db3de990..01c97dc37 100644
--- a/pngrutil.c
+++ b/pngrutil.c
@@ -3163,12 +3163,27 @@ png_check_chunk_length(png_const_structrp png_ptr, png_uint_32 length)
    if (png_ptr->chunk_name == png_IDAT)
    {
       png_alloc_size_t idat_limit = PNG_UINT_31_MAX;
+#ifdef MAGMA_ENABLE_FIXES
       size_t row_factor =
          (size_t)png_ptr->width
          * (size_t)png_ptr->channels
          * (png_ptr->bit_depth > 8? 2: 1)
          + 1
          + (png_ptr->interlaced? 6: 0);
+#else
+      size_t row_factor_l =
+         (size_t)png_ptr->width
+         * (size_t)png_ptr->channels
+         * (png_ptr->bit_depth > 8? 2: 1)
+         + 1
+         + (png_ptr->interlaced? 6: 0);
+
+#ifdef MAGMA_ENABLE_CANARIES
+      MAGMA_LOG("AAH001", row_factor_l == ((size_t)1 << (sizeof(png_uint_32) * 8)));
+#endif
+
+      size_t row_factor = (png_uint_32)row_factor_l;
+#endif
       if (png_ptr->height > PNG_UINT_32_MAX/row_factor)
          idat_limit = PNG_UINT_31_MAX;
       else

The patch shows that a bug is located in the pngrutil.c file, in function png_check_chunk_length starting at line 3163. The lines starting with + constitute the bug. Indeed, according to the CVE-2018-13785 which corresponds to this patch, "a wrong calculation of row_factor in the png_check_chunk_length function (pngrutil.c) may trigger an integer overflow and resultant divide-by-zero while processing a crafted PNG file, leading to a denial of service". Hence, using the Magma *.patch files you can easily locate the corresponding bugs.

TLDR; (Summary)

Magma ground-truth fuzzing benchmark is a collection of popular and diverse open-source libraries with 118 real-world front-ported bugs available in the form of Git patch files under targets/<target_name>/patches/bugs/*.patch. While the EPFL HexHive team originally designed Magma for benchmarking fuzzer performance, it can be used to evaluate the effectiveness of any method for discovering security vulnerabilities in software since patches give you the necessary ground truth.

How to Backup MariaDB Database Running in a Kubernetes Pod

Paul Duplys — Fri, 11 Sep 2020 19:54:33 +0000

The easiest way to backup (and restore) a MariaDB database is to use the mysqldump tool. This is especially true when you don't have a huge amount of data in your database. mysqldump dumps the data in SQL format which is really handy as it can be imported by most database engines. For more details, see this MariaDB documentation.

But how do you backup a database running in a container? First, get the name of the Kubernetes Pod containing your database:

ubuntu@ubuntu:~$ microk8s.kubectl get all
NAME                                 READY   STATUS    RESTARTS   AGE
pod/mediawiki-app-55f45cf568-gmpzv   1/1     Running   2          5d22h
pod/mediawiki-db-5cb8db589f-r6q8k    1/1     Running   0          5d23h
pod/my-nginx-9b596c8c4-4jp7d         1/1     Running   15         102d
pod/my-nginx-9b596c8c4-fnlm7         1/1     Running   2          12d
pod/my-nginx-9b596c8c4-hmz4r         1/1     Running   2          12d

NAME                       TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)          AGE
service/kubernetes         ClusterIP   10.152.183.1     <none>        443/TCP          104d
service/mediawiki-db-srv   NodePort    10.152.183.195   <none>        3306:31501/TCP   5d23h
service/mediawiki-srv      NodePort    10.152.183.17    <none>        80:32681/TCP     28d
service/my-nginx-np        NodePort    10.152.183.73    <none>        80:30178/TCP     102d

NAME                            READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/mediawiki-app   1/1     1            1           11d
deployment.apps/mediawiki-db    1/1     1            1           5d23h
deployment.apps/my-nginx        3/3     3            3           102d

NAME                                       DESIRED   CURRENT   READY   AGE
replicaset.apps/mediawiki-app-5494668f87   0         0         0       5d23h
replicaset.apps/mediawiki-app-55f45cf568   1         1         1       5d23h
replicaset.apps/mediawiki-app-75cb9c97d7   0         0         0       11d
replicaset.apps/mediawiki-db-5cb8db589f    1         1         1       5d23h
replicaset.apps/my-nginx-9b596c8c4         3         3         3       102d
ubuntu@ubuntu:~$

In my setup, it's the Pod mediawiki-db-5cb8db589f-r6q8k (I know this because in my Kubernetes Deployment manifest file, I named my MariaDB Deployment mediawiki-db). Now that we know the Deployment name, we need to find out its IP address. You can easily do this with the kubectl describe command (note that since I'm running Canonical's microk8s, a lightweight Kubernetes variant, and I don't have shell aliases, I need to prepend microk8s.; if you are running regular Kubernetes, just use the standard kubectl command):

ubuntu@ubuntu:~$ microk8s.kubectl describe pod mediawiki-db-5cb8db589f-r6q8k
Name:         mediawiki-db-5cb8db589f-r6q8k

# -- snip --

IP:           10.1.49.32
IPs:
  IP:           10.1.49.32
Controlled By:  ReplicaSet/mediawiki-db-5cb8db589f

# -- snip --

Events:          <none>

You can double check that the IP address is correct by issuing a simple ping command:

ubuntu@ubuntu:~$ ping 10.1.49.32
PING 10.1.49.32 (10.1.49.32) 56(84) bytes of data.
64 bytes from 10.1.49.32: icmp_seq=1 ttl=63 time=2.04 ms
64 bytes from 10.1.49.32: icmp_seq=2 ttl=63 time=0.615 ms
^C
--- 10.1.49.32 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 0.615/1.329/2.044/0.715 ms
ubuntu@ubuntu:~$

Now that you know the IP address exists and is alive, you can check whether the MariaDB database is actually running. To do this, you can use the mysql command to connect to a MariaDB database on that Pod. In my case, wikiuser is the user name for the database, 10.1.49.32 is the IP address of the corresponding Kubernetes Pod, and my_wiki is the name of the database I have previously created.

ubuntu@ubuntu:~$ mysql -u wikiuser -p -h 10.1.49.32 my_wiki
Enter password: 
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 160
Server version: 5.5.5-10.3.22-MariaDB-0+deb10u1 Raspbian 10

Copyright (c) 2000, 2020, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> exit
Bye
ubuntu@ubuntu:~$

If all looks well, you can issue the mysqldump command as shown below. The command takes the username (-u switch), the -p switch for password (if you omit the password value, mysqldump prompts for one), the IP address (-h switch), and the name of the database to be dumped. By default, the mysqldump outputs the SQL statements to create and populate the database to the standard output. Hence, to backup into a file, you can simply redirect the output to an arbitrary file or your choice using the > shell operator:

ubuntu@ubuntu:~$ mysqldump -u wikiuser -p -h 10.1.49.32 my_wiki > my_wiki_backup.sql
Enter password:  
ubuntu@ubuntu:~$ head -n 20 my_wiki_backup.sql 
-- MySQL dump 10.13  Distrib 5.7.31, for Linux (aarch64)
--
-- Host: 10.1.49.32    Database: my_wiki
-- ------------------------------------------------------
-- Server version   5.5.5-10.3.22-MariaDB-0+deb10u1

/*!40101 SET @OLD_CHARACTER_SET_CLIENT=@@CHARACTER_SET_CLIENT */;
/*!40101 SET @OLD_CHARACTER_SET_RESULTS=@@CHARACTER_SET_RESULTS */;
/*!40101 SET @OLD_COLLATION_CONNECTION=@@COLLATION_CONNECTION */;
/*!40101 SET NAMES utf8 */;
/*!40103 SET @OLD_TIME_ZONE=@@TIME_ZONE */;
/*!40103 SET TIME_ZONE='+00:00' */;
/*!40014 SET @OLD_UNIQUE_CHECKS=@@UNIQUE_CHECKS, UNIQUE_CHECKS=0 */;
/*!40014 SET @OLD_FOREIGN_KEY_CHECKS=@@FOREIGN_KEY_CHECKS, FOREIGN_KEY_CHECKS=0 */;
/*!40101 SET @OLD_SQL_MODE=@@SQL_MODE, SQL_MODE='NO_AUTO_VALUE_ON_ZERO' */;
/*!40111 SET @OLD_SQL_NOTES=@@SQL_NOTES, SQL_NOTES=0 */;

--
-- Table structure for table `actor`
--
ubuntu@ubuntu:~$

Building an Exocortex: Mediawiki on a microk8s Kubernetes Cluster

Paul Duplys — Wed, 29 Jul 2020 07:03:49 +0000

Boost Your Memory

In a book I read some time ago, I stumbled upon the idea of artificially augmenting one's memory by systematically storing everything one learns in a single place. The book used the term exocortex to refer to this concept of extending the natural brain's memory capabilities by external technical means. Since then, I have used such a mechanism to keep everything I consider important (mostly technical) stuff in one place.

In this post, I'll show you how you can use a MediaWiki instance on your private Raspberry Pi Kubernetes cluster to implement an exocortex. MediaWiki is an open source wiki software originally authored by Magnus Manske and improved by Lee Danial Crocker. The most prominent website running MediaWiki is Wikipedia. It is written in the PHP programming language and stores the contents into a database.

For ease of administration, we'll use Kubernetes to deploy the Mediawiki application. Kubernetes will ensure that things work reliably. For example, if the applications stops working, Kubernetes will immediately spawn a new instance.

A word on spelling: for easier reading -- and in alignment with conventions used in the official Kubernetes documentation -- I will capitalize the names of the Kubernetes objects. Hence, the words "Deployment" and "Pod" refer to specific Kubernetes objects (more details on that later).

The Ingredients

Here's a list of ingredients we'll need:

A microk8s Kubernetes cluster on Raspberry Pi
The official mediawiki Docker image
The official mariadb Docker image
A docker-compose.yml file that we will use as a starting point

Starting Point

If you look up the official MediaWiki Docker image on DockerHub, you will also find there a docker-compose file recommended to deploy the dockerized MediaWiki. We will use this file as a starting point for writing our Kubernetes manifests so we can deploy MediaWiki using microk8s.kubectl apply on the Raspberry Pi Kubernetes cluster. This is what the docker-compose file looks like:

# MediaWiki with MariaDB
#
# Access via "http://localhost:8080"
#   (or "http://$(docker-machine ip):8080" if using docker-machine)
version: '3'
services:
  mediawiki:
    image: mediawiki
    restart: always
    ports:
      - 8080:80
    links:
      - database
    volumes:
      - /var/www/html/images
      # After initial setup, download LocalSettings.php to the same directory as
      # this yaml and uncomment the following line and use compose to restart
      # the mediawiki service
      # - ./LocalSettings.php:/var/www/html/LocalSettings.php
  database:
    image: mariadb
    restart: always
    environment:
      # @see https://phabricator.wikimedia.org/source/mediawiki/browse/master/includes/DefaultSettings.php
      MYSQL_DATABASE: my_wiki
      MYSQL_USER: wikiuser
      MYSQL_PASSWORD: example
      MYSQL_RANDOM_ROOT_PASSWORD: 'yes'

Essentially, it consists of two services: the mediawiki application itself and the database service for storing the data. The mediawiki service is exposed to port 8080 and requires two volumes for regular operation. The database service run a MariaDB container and needs several environment variables to be set so that the mediawiki application can access the database. In the remainder of this post, we'll recreated this setup using Kubernetes mechanisms.

The MediaWiki Application

Let's start by creating a microk8s Kubernetes deployment for the MediaWiki container.

The Minimal MediaWiki Deployment

A Kubernetes Deployment is a Kubernetes object — a persistent entity in the Kubernetes system defining the desired state — that basically specifies a ReplicaSet and the replicated Pods to be run. The specification of a Kubernetes object, referred to as a manifest, must contain the following required fields:

apiVersion specifying which version of the Kubernetes API is used to create the object
kind specifying the kind of the object to be created
metadata for uniquely identifying the object (at least a name string and a UID)
spec specifying the object's desired state

Therefore, as a minimum, the MediaWiki Deployment manifest would look something like this (feel free to choose your own UIDs and label values):

apiVersion: apps/v1
kind: Deployment
metadata:
  name: mediawiki-app
spec:
  selector:
    matchLabels:
      app: mediawiki-app
  replicas: 2
  template:
    metadata:
      labels:
        app: mediawiki-app
    spec:
      containers:
        - name: mediawiki-container
          image: mediawiki

Note that the label in spec.selector.matchLabels must match the label in spec.template.metadata.labels. This is because the .spec.selector field defines how the Deployment finds which Pods to manage. Thus, you need to select a label that is defined in the Pod template (app: mediawiki-app).

We can create the Deployment by issuing microk8s.kubectl apply -f mediawiki.yml on the command line of the Raspberry Pi (assuming you saved the Deployment manifest in the file mediawiki.yml). It takes some time when we run it for the first time because the images must be downloaded first. After a while — it takes approximately 5-10 minutes on my RPi — you should see something like this:

ubuntu@ubuntu:~/mediawiki$ microk8s.kubectl get all
NAME                                 READY   STATUS    RESTARTS   AGE
pod/mediawiki-app-656f6f8d64-7f4xz   1/1     Running   0          11m
pod/mediawiki-app-656f6f8d64-9x924   1/1     Running   0          11m

NAME                  TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)        AGE
service/kubernetes    ClusterIP   10.152.183.1    <none>        443/TCP        75d

NAME                            READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/mediawiki-app   2/2     2            2           11m

NAME                                       DESIRED   CURRENT   READY   AGE
replicaset.apps/mediawiki-app-656f6f8d64   2         2         2       11m
ubuntu@ubuntu:~/mediawiki$

The Deployment is running, the ReplicaSet was created and the two Pods are running just like we specified in the above manifest.

Adding Volumes

In the docker-compose file, our starting point from above, the mediawiki service uses a Docker volume for the /var/www/html/images directory within the mediawiki container. So let's add a volume to our Kubernetes Deployment. The official documentation for Kubernetes volumes can be found here. It turns out, we need to add a volumeMounts field to spec.containers and pass it a list (that's why the next line starts with a -) containing the mountPath and the name of the volume. We also need to add the a volume field to spec and list the volume we want to add. The Deployment manifest now looks like this:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: mediawiki-app
spec:
  selector:
    matchLabels:
      app: mediawiki-app
  replicas: 2
  template:
    metadata:
      labels:
        app: mediawiki-app
    spec:
      containers:
        - name: mediawiki-container
          image: mediawiki
          volumeMounts:
            - mountPath: /var/www/html/images
              name: mediawiki-volume
      volumes:
        - name: mediawiki-volume

To update our MediaWiki Deployment, we issue again the same command microk8s.kubectl apply -f mediawiki.yml. If we now take a closer look at one of the Pods, we should see a volume attached to it (the actual output of microk8s.kubectl describe pod is very detailed, I have trimmed it here for the ease of reading):

ubuntu@ubuntu:~/mediawiki$ microk8s.kubectl describe pod mediawiki-app-656f6f8d64-7f4xz
Name:         mediawiki-app-656f6f8d64-7f4xz

# -- snip --

Containers:
  mediawiki-container:
    Container ID:   containerd://8525869985915306078e1c2b836682ddf145510a4fe04b7155618a749ed87e2d
    Image:          mediawiki

    # -- snip --

    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-krfdn (ro)
      /var/www/html/images from mediawiki-volume (rw)

# -- snip --

Volumes:
  mediawiki-volume:
    Type:       EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:     
    SizeLimit:  <unset>

# -- snip --

ubuntu@ubuntu:~/mediawiki$

Note that microk8s reports that this volume is just a temporary directory. This is because this type of volume is not persistent in the sense that it is deleted if the pod ceases to exist.

Restart

Our docker-compose file defines the restart policy always for the mediawiki service. As described in the official Kubernetes documentation for the Pod lifecycle, a Pod specification has a restartPolicy field with possible values Always, OnFailure, and Never. The default value is Always. So we're done here!

Links

In docker-compose the link directive creates a link to containers in another service. From the official Docker Compose documentation:

Containers for the linked service are reachable at a hostname identical to the alias, or the service name if no alias was specified.

Links are not required to enable services to communicate - by default, any service can reach any other service at that service's name. (See also, the Links topic in Networking in Compose.)

Links also express dependency between services in the same way as depends_on, so they determine the order of service startup.

I'm not sure whether I need this in Kubernetes because the official documentation says:

Kubernetes gives every pod its own cluster-private IP address, so you do not need to explicitly create links between pods or map container ports to host ports. This means that containers within a Pod can all reach each other's ports on localhost, and all pods in a cluster can see each other without NAT.

Ports

The mediawiki service in the docker-compose file exposes port 80 of the mediawiki container and maps it to the port 8080 on the host. To replicate this in our Kubernetes manifest, we first add the spec.containers.ports field and pass it a list of the container ports to be exposed (- containerPort: 80 in this case). The manifest now looks like this:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: mediawiki-app
spec:
  selector:
    matchLabels:
      app: mediawiki-app
  replicas: 2
  template:
    metadata:
      labels:
        app: mediawiki-app
    spec:
      containers:
        - name: mediawiki-container
          image: mediawiki
          volumeMounts:
            - mountPath: /var/www/html/images
              name: mediawiki-volume
          ports:
            - containerPort: 80
      volumes:
        - name: mediawiki-volume

We again update our Deployment using the microk8s.kubectl apply -f mediawiki.yml command. We can verify that ports specification took effect:

ubuntu@ubuntu:~/mediawiki$ microk8s.kubectl describe deployment mediawiki-app
Name:                   mediawiki-app

# -- snip --

Pod Template:
  Labels:  app=mediawiki-app
  Containers:
   mediawiki-container:
    Image:        mediawiki
    Port:         80/TCP
    Host Port:    0/TCP
    Environment:  <none>

# -- snip --

ubuntu@ubuntu:~/mediawiki$ microk8s.kubectl describe pod mediawiki-app-5494668f87-5xldp
Name:         mediawiki-app-5494668f87-5xldp

# -- snip --

Containers:
  mediawiki-container:
    Container ID:   containerd://6ba69454dc74ebc359466edc233197e2cf11f3ae8365d4fcf5ea60b0b62ec3b7
    Image:          mediawiki
    Image ID:       docker.io/library/mediawiki@sha256:e3be6a44c1d82e454657a013c5df29f7a9a0bfc112325d6d9df1ab1e21087b51
    Port:           80/TCP
    Host Port:      0/TCP

# -- snip --

ubuntu@ubuntu:~/mediawiki$

As you can see, port 80 of the MediaWiki container is now exposed. What this means is that your Pod is now accessible through port 80 from any Kubernetes Node within your cluster.

Adding a MediaWiki Service

But what happens if a Node dies? The Pods running on that Node die with it. The Deployment will create new Pods, but they will have different IP addresses. And that's exactly the problem solved by a Kubernetes Service.

A Kubernetes Service defines a logical set of Pods that provide the same functionality and run somewhere in the Kubernetes cluster. Upon creation, each Service is assigned a unique IP address, the so-called clusterIP. This address will not change while the Service is alive. Communication to the Service will be automatically load-balanced to some Pod that is a member of that Service.

Thus, we need to create a Service to expose the mediawiki pods to the outside world (outside of the Kubernetes cluster, that is). As with our MediaWiki Deployment, we store the Service manifest in a new file mediawiki-service.yml:

apiVersion: v1
kind: Service
metadata:
  name: mediawiki-srv
  labels:
    app: mediawiki-app
spec:
  ports:
    - port: 8883
      protocol: TCP
  selector:
    app: mediawiki-app

Note that the Service label must match our selector label in the Deployment spec (in our case the label app: mediawiki-app). By the way: for some odd reason, microk8s.kubectl returns an error saying Service is an unknown type if I use apiVersion: apps/v1 as in the Deployment (using just v1 instead works).

To start the Service, we issue the microk8s.kubectl apply -f mediawiki-service.yml command. Let's now inspect out Kubernetes cluster:

ubuntu@ubuntu:~/mediawiki$ microk8s.kubectl get all
NAME                                 READY   STATUS    RESTARTS   AGE
pod/mediawiki-app-5494668f87-5xldp   1/1     Running   0          19m
pod/mediawiki-app-5494668f87-p7p4s   1/1     Running   0          19m

NAME                    TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)        AGE
service/kubernetes      ClusterIP   10.152.183.1    <none>        443/TCP        75d
service/mediawiki-srv   ClusterIP   10.152.183.17   <none>        8883/TCP       3m51s

NAME                            READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/mediawiki-app   2/2     2            2           49m

NAME                                       DESIRED   CURRENT   READY   AGE
replicaset.apps/mediawiki-app-5494668f87   2         2         2       19m
replicaset.apps/mediawiki-app-656f6f8d64   0         0         0       49m
ubuntu@ubuntu:~/mediawiki$

From the above output, you can see that the Service was created. However, the Service has the type ClusterIP meaning that it cannot be reached from outside of the Kubernetes cluster. To fix this, we need to change the type of the mediawiki-srv service into NodePort. In addition, the port should be 80 (as this is the port exposed by the mediawiki Pod):

apiVersion: v1
kind: Service
metadata:
  name: mediawiki-srv
  labels:
    app: mediawiki-app
spec:
  type: NodePort
  ports:
    - port: 80
      protocol: TCP
  selector:
    app: mediawiki-app

We microk8s.kubectl apply the Service manifest again and check the result:

ubuntu@ubuntu:~/mediawiki$ microk8s.kubectl apply -f mediawiki-service.yml 
service/mediawiki-srv configured
ubuntu@ubuntu:~/mediawiki$ microk8s.kubectl get all
NAME                                 READY   STATUS    RESTARTS   AGE
pod/mediawiki-app-5494668f87-5xldp   1/1     Running   0          30m
pod/mediawiki-app-5494668f87-p7p4s   1/1     Running   0          30m

NAME                    TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)        AGE
service/kubernetes      ClusterIP   10.152.183.1    <none>        443/TCP        76d
service/mediawiki-srv   NodePort    10.152.183.17   <none>        80:32681/TCP   14m

NAME                            READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/mediawiki-app   2/2     2            2           59m

NAME                                       DESIRED   CURRENT   READY   AGE
replicaset.apps/mediawiki-app-5494668f87   2         2         2       30m
replicaset.apps/mediawiki-app-656f6f8d64   0         0         0       59m
ubuntu@ubuntu:~/mediawiki$

Verifying the Intermediate Result

We can now check our MediaWiki Service and Deployment using the curl command line tool directly on the Raspberry Pi:

ubuntu@ubuntu:~/mediawiki$ curl http://127.0.0.1:32681
<!DOCTYPE html>
<html lang="en" dir="ltr">
    <head>
        <meta charset="UTF-8" />
        <title>MediaWiki 1.34.2</title>
        <style media="screen">
            body {
                color: #000;
                background-color: #fff;
                font-family: sans-serif;
                text-align: center;
            }

            h1 {
                font-size: 150%;
            }
        </style>
    </head>
    <body>
        <img src="/resources/assets/mediawiki.png" alt="The MediaWiki logo" />

        <h1>MediaWiki 1.34.2</h1>
        <div class="error">


            <p>LocalSettings.php not found.</p>


                <p>Please <a href="/mw-config/index.php">set up the wiki</a> first.</p>


        </div>
    </body>
</html>
ubuntu@ubuntu:~/mediawiki$

Now that it works locally, we can try to access the mediawiki service from our host system, i.e., outside the Raspberry Pi. If you open your web browser and type http://<ip address of you RPi>:32681, you should see this:

The Database Application

Next, we need to create a Deployment and Service for the database (MariaDB in our case). In the docker-compose.yml file, the MariaDB Service looks like this:

version: '3'
services:

  # -- snip --

  database:
    image: mariadb
    restart: always
    environment:
      # @see https://phabricator.wikimedia.org/source/mediawiki/browse/master/includes/DefaultSettings.php
      MYSQL_DATABASE: my_wiki
      MYSQL_USER: wikiuser
      MYSQL_PASSWORD: example
      MYSQL_RANDOM_ROOT_PASSWORD: 'yes'

The MariaDB Deployment

We already saw that we don't need to do anything explicitly for the always restart policy since it's the default in Kubernetes. However, we have to use a different Docker image for MariaDB because that Docker container will run on our Raspberry Pi.

Why do we need a different Docker image? Well, the Raspberry Pi CPU is based on the ARM architecture which is a different from that of your standard PC or laptop (which nowadays mostly happens to be x86_64). A Docker image is essentially a collection of layers (packaged in *.tar files) containing, among other things, binaries needed to run your application, e.g., command utilities like cp or ls, compilers like gcc, scripting languages like python or ruby, and your application itself, say, nginx (if you're curious, you can just run docker image save <image-name> -o <some-name>.tar and look into that .tar archive to see the layers and the binaries in each layer).

If the Docker image was built on an x86_64 architecture, the resulting binaries cannot be executed on ARM architecture. Thus, many of the official Docker images from DockerHub — an the official MariaDb image happens to be one of those — will not run on a Raspberry Pi. To deal with this, you can either build your own MariaDB image for RPi or use e.g., the rpi-mariadb image, which is a port of the official mariadb image for Raspberry Pi.

A word of caution: For security reasons, you should always be very careful when using unofficial Docker images from DockerHub! Since Docker itself runs with root permissions on your system, every Docker container can essentially also access your system with root permissions (unless you have configured your Docker installation to avoid this particular threat). If you want to learn more about this, a very accessible source is the "Docker in Practice" book by Ian Miell and Aidan Hobson Sayers, Chapter 14 "Docker and Security").

Finally, we will need to define the 4 environment variables for the MariaDB container. In Kubernetes, you set environment variables using the spec.containers.env field. So the Deployment manifest for MariaDB looks like this:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: mediawiki-db
spec:
  selector:
    matchLabels:
      app: mediawiki-db
  replicas: 1
  template:
    metadata:
      labels:
        app: mediawiki-db
    spec:
      containers:
        - name: mediawiki-db-container
          image: jsurf/rpi-mariadb
          env:
            - name: MYSQL_DATABASE
              value: my_wiki
            - name: MYSQL_USER
              value: wikiuser
            - name: MYSQL_PASSWORD
              value: example
            - name: MYSQL_RANDOM_ROOT_PASSWORD
              value: 'yes'

Use the microk8s.kubectl apply -f <mariadb-deployment-manifest.yml> to create the Deployment. You can also check the DefaultSettings.php file to see why we need to set these environment variables in the MariaDB container for it to work.

Adding MariaDB Service

Like with MediaWiki itself, we need to add a Kubernetes Service for MariaDB so it is exposed on the local host and thus can be accessed by the MediaWiki application. The default port number for MariaDB is 3306 and this is also the port MediaWiki will try to connect to. Hence, the manifest for the MariaDB service looks like this:

apiVersion: v1
kind: Service
metadata:
  name: mediawiki-db-srv
  labels:
    app: mediawiki-db
spec:
  type: NodePort
  ports:
    - port: 3306
      protocol: TCP
  selector:
    app: mediawiki-db

Use the microk8s.kubectl apply -f <mariadb-service-manifest.yml> to create the Service. You now should see the following Kubernetes objects running:

ubuntu@ubuntu:~/mediawiki$ microk8s.kubectl get all
NAME                                 READY   STATUS    RESTARTS   AGE
pod/mediawiki-app-55f45cf568-gmpzv   1/1     Running   0          9h
pod/mediawiki-db-5cb8db589f-r6q8k    1/1     Running   0          9h

NAME                       TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)          AGE
service/kubernetes         ClusterIP   10.152.183.1     <none>        443/TCP          98d
service/mediawiki-db-srv   NodePort    10.152.183.195   <none>        3306:31501/TCP   9h
service/mediawiki-srv      NodePort    10.152.183.17    <none>        80:32681/TCP     9h

NAME                            READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/mediawiki-app   1/1     1            1           9h
deployment.apps/mediawiki-db    1/1     1            1           9h

NAME                                       DESIRED   CURRENT   READY   AGE
replicaset.apps/mediawiki-app-55f45cf568   1         1         1       9h
replicaset.apps/mediawiki-db-5cb8db589f    1         1         1       9h
ubuntu@ubuntu:~/mediawiki$

Setting Up MediaWiki

From your host computer, you can now point your browser to http://<ip address of you RPi>:32681 and click on the link to start setting up the MediaWiki application. The installation procedure is self-explanatory. The only thing you need to adjust during the setup is to set the IP address of the database to 127.0.0.1 (instead of the default localhost).

At the end of the installation, you will be prompted to download the LocalSettings.php file. You will need this setting file to run MediaWiki (otherwise, you will always end up in the setup mode). First, you need to scp this file to your Raspberry Pi:

paulduplys@Pauls-MBP Downloads % paulduplys@Pauls-MBP Downloads % scp LocalSettings.php ubuntu@192.168.178.66:/home/ubuntu/mediawiki/
ubuntu@192.168.178.66's password: 
LocalSettings.php                                                                                                                                                                                           100% 4233   858.3KB/s   00:00    
paulduplys@Pauls-MBP Downloads %

To mount this file into the mediawiki container, we use the hostPath volume which mounts a file or directory from the host node's filesystem into your Pod. So, we need to add the following to our Deployment manifest:


# -- snip --

    spec:
      containers:
        - name: mediawiki-container
          image: mediawiki
          volumeMounts:

            # -- snip --

            - mountPath: /var/www/html/LocalSettings.php
              name: mediawiki-localsettings
          ports:
            - containerPort: 80

      # -- snip --

      volumes:
        - name: mediawiki-volume
        - name: mediawiki-localsettings
          hostPath: 
            path: /home/ubuntu/mediawiki/LocalSettings.php

The final MediaWiki Deployment manifest looks like this:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: mediawiki-app
spec:
  selector:
    matchLabels:
      app: mediawiki-app
  replicas: 1
  template:
    metadata:
      labels:
        app: mediawiki-app
    spec:
      containers:
        - name: mediawiki-container
          image: mediawiki
          volumeMounts:
            - mountPath: /var/www/html/images
              name: mediawiki-volume
            - mountPath: /var/www/html/LocalSettings.php
              name: mediawiki-localsettings
          ports:
            - containerPort: 80
      volumes:
        - name: mediawiki-volume
        - name: mediawiki-localsettings
          hostPath:
            path: /home/ubuntu/mediawiki/LocalSettings.php
            type: File

Issue microk8s.kubectl apply -f mediawiki.yml one last time and you should have a Kubernetes-managed MediaWiki instance. Enjoy!

À la Carte for Devs: microk8s on Raspberry Pi 4

Paul Duplys — Wed, 24 Jun 2020 20:58:14 +0000

After finally getting my hands on a gorgeous Raspberry Pi 4 (the version with 4 GB of RAM), I decided to test-drive it by installing Canonical's microk8s, a small, single-package, CNCF certified fully-conformant upstream Kubernetes deployment for offline development, prototyping, and testing.

It came in handy that I had watched the superb "Docker Mastery: with Kubernetes + Swarm" course by Bret Fisher and so could apply the things I learned there. I can definitely recommend this course if you are interested in learning Docker, Swarm, and Kubernetes from the ground up. Bret is a great instructor and, besides the videos and slides, the course contains a lot of good exercises and handy reference material. All in all, it took me about 2 hours to install everything and maybe another 1 hour to play around a bit and set things up.

In this post, I'll show you step by step how you can replicate that and create a microk8s Kubernetes deployment on your Raspberry Pi 4. This will give you a nice environment where you can quickly deploy any containerized applications to experiment, prototype, test, and play around at home.

The Hardware

In addition to the Raspberry Pi 4 microcomputer itself, you'll need 3 extra hardware items.

A power supply for the Raspberry. I use Raspberry Pi 15.3W USB-C, the officially recommended USB-C power supply for Raspberry Pi 4.
An microSD card for storing the operating system, Docker + Kubernetes and your applications on your Raspberry. I use a 128GB microSD card from SanDisk.
A network cable to connect the Raspberry to your router. Any Ethernet cable will do.

Installing Ubuntu 18.04 on Raspberry Pi 4

First of all, you need to install an operating system on the Raspberry. I chose Ubuntu Server because of its comprehensive package repositories and long-term support (LTS). Shortly after I was preparing this post, Ubuntu switched from 18.04 to 20.04 LTS release, but the instructions given here should work on 20.04 as well.

To get the Ubuntu image for Raspberry Pi, visit the official Ubuntu download website and download the 64-bit version of the Ubuntu Pi image. Follow the installation instructions.

Once the download is complete you need to extract the Ubuntu Server 20.04 image (the .xz file is an archive) and copy it onto your microSD card. The installation instructions contain links to short tutorials on how to do this under Linux, Windows and Macs. If you happen to be on a Windows machine, you can use the win32disk tool to write the Ubuntu image onto the microSD card (remember that you first need unpack the .xz archive).

Because I'm too lazy to use an extra screen for the installation, I rather chose to ssh into the Raspberry Pi once it has booted. To enable this, you can follow the instructions described here and simply create a new file called ssh in the root directory of your microSD card (it's the same directory where files like config.txt are located). This will enable the ssh daemon at the first boot of your Rapsberry so you will be able to immediately ssh into it.

Next, you need connect the Raspberry to your router using a network cable, insert the microSD card into the SD card slot and power up the Raspberry Pi.

Connect to your router - most of them like e.g., the AVM FritzBox have a web admin frontend where you can login; If you happen to have a FritzBox, just fire up your browser and enter http://fritz.box. This will give you a login mask. Once you logged in, look up the IP address of the newly connected Raspberry Pi. Typically, it will have ubuntu as its device name.

Now, from your host (which obviously needs to be connected to the same router, e.g., via WiFi), do:

$ ssh ubuntu@<IP address of your pi>

and enter ubuntu when asked for password. When logging in for the first time, you will be asked to choose a new password. From then on, you login using the user ubuntu and your new password.

Installing `microk8s`

To install and use microk8s, you need cgroups. By default, cgroups are not enabled on the Ubuntu server image for Raspberry Pi. As described in this microk8s documentation, you need to edit the boot parameters in nobtcmd.txt (attention: in the older Pi versions before Pi 4, this file was called cmdline.txt; if you google for how to enable cgroups on a Raspberry Pi, you might hit a tutorial that uses this outdated name). Thus, fire up the vi editor as root using sudo:

ubuntu@ubuntu:~$ sudo vi /boot/firmware/nobtcmd.txt

and add the following line:

cgroup_enable=memory cgroup_memory=1

After the change, you need to reboot the Raspberry Pi for it to take effect. To do this, issue:

ubuntu@ubuntu:~$ sudo reboot

Next, follow the instructions on the microk8s website on how to install microk8s using snap. As I'm writing this post, the current channel version is 1.18. So on your Raspberry Pi, issue:

ubuntu@ubuntu:~$ sudo snap install microk8s --classic --channel=1.18/stable

After the installation is complete, you need to add your current user ubuntu to the microk8s group and make some other changes. See the microk8s documentation for details. You need to do the following:

ubuntu@ubuntu:~$ sudo usermod -a -G microk8s $USER
ubuntu@ubuntu:~$ sudo chown -f -R $USER ~/.kube
ubuntu@ubuntu:~$ su - $USER
Password: 
ubuntu@ubuntu:~$ microk8s status --wait-ready
microk8s is running
addons:
dashboard: disabled
dns: disabled
helm: disabled
helm3: disabled
ingress: disabled
kubeflow: disabled
metallb: disabled
metrics-server: disabled
rbac: disabled
registry: disabled
storage: disabled
ubuntu@ubuntu:~$ microk8s kubectl get nodes
NAME     STATUS   ROLES    AGE   VERSION
ubuntu   Ready    <none>   10m   v1.18.0
ubuntu@ubuntu:~$ microk8s kubectl get services
NAME         TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)   AGE
kubernetes   ClusterIP   10.152.183.1   <none>        443/TCP   10m

Congrats! You now have a working microk8s installation on your Raspberry Pi 4!

Deploying nginx with microk8s

To play around with your new shiny microk8s installation, you can create a deployment for nginx, a popular HTTP and reverse proxy server. If you want to learn more about nginx, visit their website. Start by creating a Kubernetes deployment using the nginx Docker image and verify that the deployment has been indeed created:

ubuntu@ubuntu:~$ microk8s.kubectl create deployment my-nginx --image nginx
deployment.apps/my-nginx created
ubuntu@ubuntu:~$ microk8s.kubectl get all
NAME                           READY   STATUS    RESTARTS   AGE
pod/my-nginx-9b596c8c4-ngd2q   1/1     Running   0          12m

NAME                 TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)   AGE
service/kubernetes   ClusterIP   10.152.183.1   <none>        443/TCP   47h

NAME                       READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/my-nginx   1/1     1            1           12m

NAME                                 DESIRED   CURRENT   READY   AGE
replicaset.apps/my-nginx-9b596c8c4   1         1         1       12m
ubuntu@ubuntu:~$ microk8s.kubectl get services
NAME         TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)   AGE
kubernetes   ClusterIP   10.152.183.1   <none>        443/TCP   47h

Next, you can optionally increase the number of your replicas (and again verify that it worked):

ubuntu@ubuntu:~$ microk8s.kubectl scale deployment my-nginx --replicas 3
deployment.apps/my-nginx scaled
ubuntu@ubuntu:~$ microk8s.kubectl get all
NAME                           READY   STATUS    RESTARTS   AGE
pod/my-nginx-9b596c8c4-4jp7d   1/1     Running   0          7s
pod/my-nginx-9b596c8c4-7ql2q   1/1     Running   0          7s
pod/my-nginx-9b596c8c4-ngd2q   1/1     Running   0          14m

NAME                 TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)   AGE
service/kubernetes   ClusterIP   10.152.183.1   <none>        443/TCP   47h

NAME                       READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/my-nginx   3/3     3            3           14m

NAME                                 DESIRED   CURRENT   READY   AGE
replicaset.apps/my-nginx-9b596c8c4   3         3         3       14m

Finally, you need to expose a so-called NodePort so that the nginx deployment can be accessed from anywhere within your home network using the IP address of your Raspberry Pi. Issue the following commands:

ubuntu@ubuntu:~$ microk8s.kubectl expose deployment my-nginx --port 80 --name my-nginx-np --type NodePort
service/my-nginx-np exposed
ubuntu@ubuntu:~$ microk8s.kubectl get services
NAME          TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)        AGE
kubernetes    ClusterIP   10.152.183.1    <none>        443/TCP        47h
my-nginx-np   NodePort    10.152.183.73   <none>        80:30178/TCP   13s
ubuntu@ubuntu:~$ microk8s.kubectl get all
NAME                           READY   STATUS    RESTARTS   AGE
pod/my-nginx-9b596c8c4-4jp7d   1/1     Running   0          3m34s
pod/my-nginx-9b596c8c4-7ql2q   1/1     Running   0          3m34s
pod/my-nginx-9b596c8c4-ngd2q   1/1     Running   0          17m

NAME                  TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)        AGE
service/kubernetes    ClusterIP   10.152.183.1    <none>        443/TCP        47h
service/my-nginx-np   NodePort    10.152.183.73   <none>        80:30178/TCP   20s

NAME                       READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/my-nginx   3/3     3            3           17m

NAME                                 DESIRED   CURRENT   READY   AGE
replicaset.apps/my-nginx-9b596c8c4   3         3         3       17m

Let's verify that nginx is working and can be accessed using the host IP address, from outside of the Raspberry Pi. To do this, exit your ssh session and from your host machine, do:

$ curl <IP address of your Raspberry Pi>:30178
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
    body {
        width: 35em;
        margin: 0 auto;
        font-family: Tahoma, Verdana, Arial, sans-serif;
    }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>

Congratulations!! You now have a working microk8s deployment and the nginx application up and running on Kubernetes.

Further References

You can also check out these references:

Growing Your Applications: Adding Nodes to a microk8s Kubernetes Cluster

Paul Duplys — Sat, 20 Jun 2020 21:01:10 +0000

With microk8s up and running, the next step is to add new nodes to the Kubernetes cluster. Adding nodes not only increases the cluster's computing resources, but also adds redundancy. Redundancy, in turn, increases the availability and reliability of the cluster because the Kubernetes Pods, e.g., the replicas of the same service, can be distributed over multiple hardware devices. Multiple hardware instances mean that failures can be tolerated. That's exactly the reason why the Kubernetes scheduler tries to ensure that Pods from the same application are distributed onto different machines.

This post is a step-by-step guide for adding new nodes to a microk8s Kubernetes cluster running on Raspberry Pi (RPi) microcomputers. In particular, we will discuss how to add RPi nodes running the Ubuntu 20.04 release. As you will see, Ubuntu 20.04 nodes require a little manual tweaking related to cgroups before you can actually use them in an microk8s Kubernetes cluster.

Note: In the remainder of this post, all shell commands are executed on the RPi minicomputers. For me, the most convenient way to manage these microcomputers is to ssh into them from my development laptop. In case you'd like to know how to enable ssh, the official Raspberry Pi documentation on remote access via ssh is a good place to start.

Luckily for us, Kubernetes makes adding new nodes really easy. In case of microk8s, the shell command microk8s add-node generates a connection string and outputs a list of suggested microk8s join commands for adding new nodes to the current microk8s cluster. The cluster node on which this command is executed becomes the main node of the Kubernetes cluster and will host its control plane.

So, to get started, connect to the main Raspberry Pi|s (i.e., the RPi you have designated to become the main node of your Kubernetes cluster) and issue:

ubuntu@ubuntu:~$ microk8s add-node
Join node with: microk8s join 192.168.178.66:25000/ipySurdsIOGAlYyYQzzOOSlpUTEDNGfk

If the node you are adding is not reachable through the default interface you can use one of the following:
 microk8s join 192.168.178.66:25000/ipySurdsIOGAlYyYQzzOOSlpUTEDNGfk
 microk8s join 10.1.79.0:25000/ipySurdsIOGAlYyYQzzOOSlpUTEDNGfk
 microk8s join 172.17.0.1:25000/ipySurdsIOGAlYyYQzzOOSlpUTEDNGfk
ubuntu@ubuntu:~$

In the above output of the microk8s add-node command you see the string ipySurdsIOGAlYyYQzzOOSlpUTEDNGfk that is required for a new node to join the Kubernetes cluster. The output also gives multiple suggestions for the parameters of the microk8s join command, which is great if you should have any issues with your setup. (Note: the value of your token will be different since Kubernetes generates it individually for every cluster).

Connect to the RPi minicomputer that you want to add to the microk8s cluster. The microk8s software must be installed and configured on this RPi before you proceed. If that's not the case, check the "Installing microk8s and nginx on Raspberry Pi 4" post for a step-by-step installation guide.

If you installed the operating system on your Raspberry Pi from a standard Linux image (in my setup, I use the Ubuntu 20.04 image for my Raspberry Pi), you might want to change the hostname of your second Raspberry Pi device to something unique (otherwise, you will see multiple Kubernetes nodes all having the same name ubuntu later on in your Kubernetes dashboard/command line). Under Linux, you can change the hostname like this:

edit /etc/hostname and replace the current hostname with the new hostname
edit /etc/hosts and replace every occurrence of the current hostname with the new hostname
reboot the system by issuing sudo reboot

In my case, the hostname of the second RPi is ubuntu-2. To add the second RPi to the microk8s cluster, I need to ssh into ubuntu-2 and issue the command:

ubuntu@ubuntu-2:~$ microk8s join 192.168.178.66:25000/ipySurdsIOGAlYyYQzzOOSlpUTEDNGfk
ubuntu@ubuntu-2:~$

You can verify that the new node has been successfully added to your Kubernetes cluster by logging into the Raspberry Pi running the main cluster node (in my setup, the RPi with the hostname ubuntu) and issuing:

ubuntu@ubuntu:~$ microk8s.kubectl get nodes
NAME       STATUS     ROLES    AGE   VERSION
ubuntu     Ready      <none>   59d   v1.18.3
ubuntu-2   NotReady   <none>   98s   v1.18.3
ubuntu@ubuntu:~$

As you can see from the output above, we now have two nodes - ubuntu and ubuntu-2 - in our microk8s Kubernetes cluster. However, we have an issue that needs to be fix: The newly added node is in the state NotReady. It turns out, this is not due to a timing issue - even after more than an hour the node remains in this state.

After searching a bit, I stumbled upon the Konstantinos Tsakalozos comment on this issue in the microk8s GitHub project.

K8s was complaining that the cgroup memory was not enabled. Looking at /proc/cgroups was showing memeory disabled. Changes in config.txt are not enough, what actually worked is editing /boot/firmware/nobtcmd.txt, appending cgroup_enable=memory cgroup_memory=1. and rebooting.

Inspecting the /proc/cgroups file on my second RPi (the one I want to add to the microk8s cluster) confirmed this:

ubuntu@ubuntu-2:~$ sudo cat /proc/cgroups 
#subsys_name    hierarchy   num_cgroups enabled
cpuset  8   1   1
cpu 7   1   1
cpuacct 7   1   1
blkio   3   1   1
memory  0   56  0
devices 5   47  1
freezer 6   2   1
net_cls 10  1   1
perf_event  9   1   1
net_prio    10  1   1
pids    2   52  1
rdma    4   1   1
ubuntu@ubuntu-2:~$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 20.04 LTS
Release:    20.04
Codename:   focal
ubuntu@ubuntu-2:~$ uname -a
Linux ubuntu-2 5.4.0-1012-raspi #12-Ubuntu SMP Wed May 27 04:08:35 UTC 2020 aarch64 aarch64 aarch64 GNU/Linux

The cgroups man page explains that Linux control groups, usually referred to as cgroups, are a Linux kernel feature which allow processes to be organized into hierarchical groups whose resource usage can then be limited and monitored. In a nutshell, as Tim Downey puts it in his post, cgroups allow you to do things like limit the amount of CPU usage of a process, limit the amount of memory that a process can consume, control how many additional processes a process can fork, and even “freeze” a process in place. In other words, cgroups are a feature provided by the Linux kernel to manage, restrict, and audit groups of processes. Compared to other approaches like the nice command or /etc/security/limits.conf, cgroups are more flexible as they can operate on (sub)sets of processes (possibly with different system users). cgroups provide a mechanism for aggregating/partitioning sets of tasks, and all their future children, into hierarchical groups with specialized behavior. A container orchestrator like Kubernetes uses cgroups to place resource limits on the containers it creates. In case you want to learn more about cgroups, check out this documentation on kernel.org.

Coming back to our issue, in contrast to the RPi running Ubuntu 20.04, the cgroups settings on my main RPi (the main Kubernetes node) look like this:

ubuntu@ubuntu:~$ sudo cat /proc/cgroups 
#subsys_name    hierarchy   num_cgroups enabled
cpuset  9   33  1
cpu 4   111 1
cpuacct 4   111 1
blkio   2   111 1
memory  3   125 1
devices 7   111 1
freezer 11  34  1
net_cls 6   33  1
perf_event  8   33  1
net_prio    6   33  1
pids    5   117 1
rdma    10  1   1
ubuntu@ubuntu:~$ uname -a
Linux ubuntu 5.3.0-1017-raspi2 #19~18.04.1-Ubuntu SMP Fri Jan 17 11:14:07 UTC 2020 aarch64 aarch64 aarch64 GNU/Linux
ubuntu@ubuntu:~$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 18.04.4 LTS
Release:    18.04
Codename:   bionic
ubuntu@ubuntu:~$

Thus, as described in this post, you have to edit the /boot/firmware/cmdline.txt file to look like this:

net.ifnames=0 dwc_otg.lpm_enable=0 console=serial0,115200 cgroup_enable=cpuset cgroup_enable=memory cgroup_memory=1 console=tty1 root=LABEL=writable rootfstype=ext4 elevator=deadline rootwait fixrtc

by adding the cgroup options:

cgroup_enable=cpuset cgroup_enable=memory cgroup_memory=1

Finally, you need to reboot the RPi for the changes to take effect. Probably the simplest way to do this is by issuing the following command :

ubuntu@ubuntu-2:~$ sudo reboot

After the reboot is complete and you have logged in into the second RPi, you can check the cgroups settings and should see the memory cgroup enabled:

ubuntu@ubuntu-2:~$ cat /proc/cgroups 
#subsys_name    hierarchy   num_cgroups enabled
cpuset  11  6   1
cpu 4   6   1
cpuacct 4   6   1
blkio   3   6   1
memory  9   101 1
devices 5   52  1
freezer 6   7   1
net_cls 7   6   1
perf_event  2   6   1
net_prio    7   6   1
pids    8   57  1
rdma    10  1   1
ubuntu@ubuntu-2:~$

As a result, if you now ssh into the Raspberry Pi running the main microk8s cluster node and check the node status in the cluster, you should see something like this:

ubuntu@ubuntu:~$ microk8s.kubectl get nodes
NAME       STATUS   ROLES    AGE     VERSION
ubuntu     Ready    <none>   59d     v1.18.3
ubuntu-2   Ready    <none>   4h12m   v1.18.3
ubuntu@ubuntu:~$

Voilà - as you can see, our microk8s Kubernetes cluster is now composed of two nodes, i.e., two RPi minicomputers. Both Kubernetes nodes are now in the Ready state. You can verify this in the Kubernetes dashboard (in case you haven't setup the dashboard yet, take a look at this post):

To recap, the three steps needed to add a RPi running Ubuntu 20.04 to your microk8s Kubernetes cluster look like this:

connect to the RPi you want to add to your microk8s Kubernetes cluster and edit the /boot/firmware/cmdline.txt file by adding cgroup_enable=cpuset cgroup_enable=memory cgroup_memory=1 to enable cgroup memory. Restart that RPi (e.g., by issuing sudo reboot)
generate the connection token on your main microk8s node using the microk8s add-node command
add the second RPi to the microk8s cluster using the microk8s join command with the token obtained in step 2

DEV Community: Paul Duplys

How to Stop Your AI from Making Things Up: A Guide to Grounding LLM Responses in Data

The Frustration is Real (and Justified)

The Solution: Ground Your Questions in Actual Data

Why Does This Work?

Enter Context Engineering

Practical Tips for Effective Context Engineering

Bottom Line

Agentic Misalignment: Why Your AI Isn't Secretly Plotting Against You

The Turing Machine

Large Language Models

How Reasoning Models Work

So, Are Reasoning Models just Turing Machines on Steroids?

Revisiting Agentic Misalignment

Demystifying the Threat

Credits

Implementing the Royal Game of Ur using Mistral's Codestral

TLDR;

Same Prompt, Different Architecture

Code Quality ...

Attempt to Fix the Code using Codestral

Trying to Fix the Bug using Codestral

Alarm!

Conclusion

Implementing the Royal Game of Ur with Mistral's Le Chat

The Royal Game of Ur

Instructions to Le Chat

Le Chat's Responses

The Generated Code

SWE-bench & SWE-bench Verified Benchmarks

SWE-bench

Test Data

Benchmark Execution

SWE-bench Verified

Can't Access a Service Container in GitHub Actions? Here's How to Fix it

Error

Now What Does That docker network create Actually Do?

The Fix

Convenient SSH: Public Key-based Authentication and Memorable Aliases for Remote Hosts

Introduction

Enabling Public Key-based Authentication

Example

Setting up Memorable Aliases for Remote Hosts

Example

References

Security Benchmarking at 1300 C

Hot, Hotter, Magma

Benchmark, anyone?

Git Patch 101

Examples, please!

TLDR; (Summary)

How to Backup MariaDB Database Running in a Kubernetes Pod

Building an Exocortex: Mediawiki on a microk8s Kubernetes Cluster

Boost Your Memory

The Ingredients

Starting Point

The MediaWiki Application

The Minimal MediaWiki Deployment

Adding Volumes

Restart

Links

Ports

Adding a MediaWiki Service

Verifying the Intermediate Result

The Database Application

The MariaDB Deployment

Adding MariaDB Service

Setting Up MediaWiki

À la Carte for Devs: microk8s on Raspberry Pi 4

The Hardware

Installing Ubuntu 18.04 on Raspberry Pi 4

Installing microk8s

Deploying nginx with microk8s

Further References

Growing Your Applications: Adding Nodes to a microk8s Kubernetes Cluster

Now What Does That `docker network create` Actually Do?

Installing `microk8s`