DEV Community: codecowboydotio

MCP server templated reports with claude

codecowboydotio — Sat, 17 Jan 2026 03:51:41 +0000

Intro

I have been fooling around a lot with ai recently, and one of the things I've found myself doing is integrating with MCP servers and then using the projects feature in claude to generate custom reports.

What is this?

I have set up a remote MCP server that pulls data from the Star Wars API (swapi). This is a remote MCP server, so I am hosting it on one of my domains. To keep this post short and on topic, I will confine myself to the claude components and not talk about the specifics of the MCP server itself. It's there, it works, and I can connect to it.

The MCP Server

The MCP server has two tools

get_swapi_character (Get character information from SWAPI by ID)
get_all_swapi_people (Get all people from SWAPI)

While this is a very simple implementation of tools within an MCP server, it serves the purpose of generating a report using an MCP and the available tools.

Claude project

I am using a claude project here to make sure that context and instructions are all self contained. This makes it easier for me to ensure that the all of the templates and history are kept together.

You can read about claude projects here: https://www.anthropic.com/news/projects

Part of creating a project in claude is creating instructions. The instructions tell claude what it needs to do whenever the project is used.

The project instructions specify that we should:

Create a document by using our template doc.
Give the project a default name.
Read the additional instructions within the template.
Don't use the default star wars brand style!!!
Ask the user if they want a document as a ppt or a doc. All of these instructions are executed when this project is used.

The two most important ones are to read the instructions within the template file, and to not use the black and yellow star wars branding (it makes the graphs undreadable).

Template file

The template file is a standard word document. It is noteworthy because it has additional instructions embedded within it.

The template uses a variable to set the project name, however it also has further prompts within it that will be read and executed upon invocation (as per the project instructions).

I have used relatively simple and straight forward prompts in my template, I am sure that you can use more complex prompts to get different functionality.

One thing that I did not try, is placement of the explicit tool calls in the template. You will note that there is an explicit tool call in the project instructions though.

Report generation

In order to generate a report, I simply open claude, navigate to the project and say generate report.

The interesting thing here is that even though the project instruction to ask the user for a preferred output is the last project instruction, it is executed first. Claude knows that it cannot move forward without this data, so asks for it first.

As part of the thought process that is used, we can see that claude has read the template, and the instructions and synthesised these. Claude also knows which tool to call. I have found where multiple MCP servers are connected, it is faster and useful to specify either the connection by name or the tool.

The next step we can see that the correct tool call is used, and that the data is returned. This is where the client (claude) makes the tool call, and gets an API response from the the backend API.

The next two steps are interesting. The first step is where the planet data is pulled out of the existing dataset that claude queried. Essentially it's a way of rearranging or re-keying the existing data. This is because the planet data is included in the person data.

The last piece of the puzzle, claude uses the inbuilt skill to export a pptx file for me to check.

The outputs are simple. There is a graph of the planet data, and a table of the people with the fields that I requested in the template. Most amazingly, the branding is NOT in traditional black and yellow.

MCP Logs

On the server side we can see that the client (claude) made a call to list the tools, and then called the get_all_swapi_people tool. I will delve more into this in another post.

2026-01-16 03:09:20 - mcp.server.lowlevel.server - INFO - Processing request of type ListToolsRequest
2026-01-16 03:10:39 - mcp.server.lowlevel.server - INFO - Processing request of type CallToolRequest
2026-01-16 03:10:39 - root - INFO - Tool called: get_all_swapi_people with arguments: {}
2026-01-16 03:10:39 - root - INFO - Fetching all people from SWAPI
2026-01-16 03:10:39 - root - INFO - Making request to: http://localhost:3000/people/
2026-01-16 03:10:39 - httpx - INFO - HTTP Request: GET http://localhost:3000/people/ "HTTP/1.1 200 OK"
2026-01-16 03:10:39 - root - INFO - Successfully fetched data from: http://localhost:3000/people/
2026-01-16 03:10:39 - root - INFO - Successfully fetched 83 people

Summary

If you're not using claude and MCP for automatic report generation (and more) you're missing out. The amount of effort to set all of this up was minimal. The template took roughly 5 minutes to create. The project instructions were similar. I did iterate the project instructions however I found that this was also minimal.

I see the use cases for this being plentiful. These include, generating reports for regulators, and any other report that you need to have regularly.

I highly recommend using MCP and claude projects for your report generation needs. It's easy, and amazingly flexible!

Github dockerfile service using AI - Part 2

codecowboydotio — Thu, 27 Nov 2025 08:10:02 +0000

Intro

I have been fooling around a lot with ai recently, and I thought I would write something about what I've been doing. There are a few things that I've been doing, and they're all fascinating.

Writing average looking code using minimum viable prompts.
Getting this average looking code refactored to be a robust service.
Generating API documentation - and having Claude fix this code.

This is part two of a small series that I have created to walk through the process I went through to get decent code.

A crazy idea - update my dockerfiles

I had a crazy idea. I thought to myself, let's write something that will go through my git repos and automagially update my dockerfiles so that the dockerfile uses a fixed but more recent version of the base image.

In my first post, I looked at the codebase, which frankly was very average, but worked. I thought to myself "I wonder how much of a refactor it would take to get a real service working".

As it turns out.... not very much.

Refactor my code ... please

I have been using Claude (not Claude code) for a little while now, and thought I might try to get it to refactor my terrible code to be well, better.

I use a simple prompt

refactor the following code to make it more robust

Then I pasted in my awful code.

What came out was generally pretty good, however, Claude initially generated another script that I had to run that was relatively static. On the plus side, my code was now object oriented, less procedural, and had a lot of features that I didn't have originally.

The biggest change here was that the code that was generated was robust and was object oriented. An example is below - the environment manager class that manages environment variables.

This is good practice, and is not something that I would have done myself.

class EnvironmentManager:
    """Manages environment variables and API keys."""

    @staticmethod
    def get_github_token() -> str:
        """Get GitHub token from environment or prompt user."""
        token = os.environ.get("GIT_PAT_AI")
        if not token:
            token = getpass.getpass("Enter API key for GitHub: ")
            os.environ["GIT_PAT_AI"] = token
        return token

    @staticmethod
    def get_anthropic_key() -> str:
        """Get Anthropic API key from environment or prompt user."""
        key = os.environ.get("ANTHROPIC_API_KEY")
        if not key:
            key = getpass.getpass("Enter API key for Anthropic: ")
            os.environ["ANTHROPIC_API_KEY"] = key
        return key

I am being honest here about what I would and wouldn't do without the assitance of another pAIr programmer.

I realised I didn't want a static python script. I wanted a service.

I want a REST service

I used the prompt below to get Claude to refactor my code to make it more robust, and so that it had a REST based interface that I could send requests to.

refactor the code to accept the repo as an input via a rest interface

This was the piece that blew me away.

The code that was generated was even better. I ended up with a full service that ran, and had a job manager and logger that would take all of the incoming requests and manage them.

I realised that I didn't know how to use this, so I also asked for example usage using the prompt

write a curl example of how to use the api

Asking for usage examples is the fastest way to get an idea of code you never wrote. It's also a great way to walk through usage.

The REST service

The codebase is now a BEAST, but it worked first go!!!

Sequence Diagram

I got claude to also generate a sequence diagram for me. It's large, but shows the interaction between the front end of the service, the github service and the user.

Let's run it

The proof is in the pudding here. I start the server on my laptop and I can see that it's running as a service on port 8000.

I placed the usage before the code walkthrough because I recognise not everyone will be interested in the code walkthrough - there is a conclusion at the end of the article.

Run the Server

I start up the server and the following is displayed on my terminal (stdout).

./git-test2.py
2025-11-27 17:18:04,515 - INFO - Initialized AI model: claude-opus-4-1-20250805
INFO:     Started server process [1966]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)

First Request - no changes

I send my first request to the service. I need to include a data section that includes my git PAT TOKEN. This is for access to the repo. I also include the repo owner, the repo name, the branch, the path of the file and a commit message.

The last thing is a dry run variable. It is possible to use the service in dry run mode where no changes will actually be made.

curl -X POST "http://localhost:8000/update-dockerfile"   -H "Content-Type: application/json"   -d '{
 "owner": "codecowboydotio",
 "repo": "swapi-json-server",
 "branch": "main",
 "github_token": "GIT_PAT_TOKEN",
 "dockerfile_path": "Dockerfile",
 "commit_message": "Updated Dockerfile FROM via AI",
 "dry_run": false
}'

The output on the console from my shell command is as follows. You can see that a job is created. The job has a unique ID, and is queued for processing.

[
  {
    "job_id":"109b7804-61c3-4f36-8b51-ea35642b41c2",
    "status":"pending",
    "message":"Job created and queued for processing",
    "timestamp":"2025-11-27T17:19:02.302591"
  }
]

Once the request is made, I can see the service fetch the contents, find the file, make the call to anthropic, and finally analyse the Dockerfile.

2025-11-27 17:19:02,305 - INFO - Fetching repository contents from https://api.github.com/repos/codecowboydotio/swapi-json-server/contents/
2025-11-27 17:19:02,831 - INFO - Downloading file from https://raw.githubusercontent.com/codecowboydotio/swapi-json-server/main/Dockerfile
2025-11-27 17:19:11,522 - INFO - HTTP Request: POST https://api.anthropic.com/v1/messages "HTTP/1.1 200 OK"
2025-11-27 17:19:11,533 - INFO - Successfully analyzed Dockerfile for base image updates

When I query the list of jobs using the /jobs endpoint, I can see that the status of the job is now completed, and there is a message that says no changes were needed.

curl -X GET http://localhost:8000/jobs  -H "Content-Type: application/json" | jq

The output shows the time, the ID of the request, and a message saying no changes were needed. Importantly, thecurrent and updated FROM lines are displayed.

[
  {
    "job_id": "109b7804-61c3-4f36-8b51-ea35642b41c2",
    "status": "completed",
    "message": "No changes needed - Dockerfile is already up to date",
    "timestamp": "2025-11-27T17:19:11.533974",
    "result": {
      "changed": false,
      "current_from": "FROM docker.io/library/ubuntu:24.04",
      "updated_from": "FROM docker.io/library/ubuntu:24.04"
    },
    "error": null
  }
]

Second request - perform an update

When I reset my Dockerfile to require a change and re-run the request, I see that there is a new job ID created.


{
  "job_id": "6691c4ac-e232-43fd-86f9-073440d32bac",
  "status": "pending",
  "message": "Job created and queued for processing",
  "timestamp": "2025-11-27T17:26:40.359356"
}

I then check my new job to see what the status is.

curl -X GET http://localhost:8000/jobs  -H "Content-Type: application/json" | jq

As I am using the /jobs endpoint, this lists ALL jobs, not an individual job. I can see my original job that had no change, and I can see my new job, which has successfully updated the dockerfile in my repository.

[
  {
    "job_id": "109b7804-61c3-4f36-8b51-ea35642b41c2",
    "status": "completed",
    "message": "No changes needed - Dockerfile is already up to date",
    "timestamp": "2025-11-27T17:19:11.533974",
    "result": {
      "changed": false,
      "current_from": "FROM docker.io/library/ubuntu:24.04",
      "updated_from": "FROM docker.io/library/ubuntu:24.04"
    },
    "error": null
  },
  {
    "job_id": "6691c4ac-e232-43fd-86f9-073440d32bac",
    "status": "completed",
    "message": "Successfully updated and committed Dockerfile",
    "timestamp": "2025-11-27T17:26:50.300875",
    "result": {
      "changed": true,
      "current_from": "FROM docker.io/library/ubuntu:20.04",
      "updated_from": "FROM docker.io/library/ubuntu:24.04",
      "dry_run": false,
      "commit_sha": "4304b404050e15293ed7d017752c252593cbc102",
      "file_url": "https://github.com/codecowboydotio/swapi-json-server/blob/main/Dockerfile"
    },
    "error": null
  }
]

We can see from the commits in the repo that it has indeed been updated!

Code Walkthrough

Let's walk through the code base, and how to use it.

The codebase is documented but did not contain an easy way to run it. For my own sanity I went and pasted in one of the example REST calls so that I could cut and paste it at any time.

#!/usr/bin/env python3
"""
Dockerfile Base Image Updater REST API

This service provides a REST API to automatically update Dockerfile base images
to their latest versions using AI analysis and commits the changes to GitHub.
"""

# curl -X POST "http://localhost:8000/update-dockerfile"   -H "Content-Type: application/json"   -d '{
#  "owner": "codecowboydotio",
#  "repo": "swapi-json-server",
#  "branch": "main",
#  "github_token": "XXX",
#  "dockerfile_path": "Dockerfile",
#  "commit_message": "Updated Dockerfile FROM via AI",
#  "dry_run": false
#}'

Libraries

There are a lot more imports of libraries this time. Claude chose FastAPI, and urllib3, but kept my original langhain and requests portions of code.

import asyncio
import base64
import json
import logging
import os
import sys
from dataclasses import dataclass, asdict
from datetime import datetime
from typing import Optional, Dict, Any, List
from enum import Enum
import uuid

import requests
from fastapi import FastAPI, HTTPException, BackgroundTasks, Depends
from fastapi.middleware.cors import CORSMiddleware
from fastapi.security import HTTPBearer, HTTPAuthorizationCredentials
from pydantic import BaseModel, validator, field_validator
from langchain.chat_models import init_chat_model
from requests.adapters import HTTPAdapter
from urllib3.util.retry import Retry

Logger

The logger configuration from the second iteration was kept.

# Configure logging
logging.basicConfig(
    level=logging.INFO,
    format='%(asctime)s - %(levelname)s - %(message)s',
    handlers=[
        logging.StreamHandler(sys.stdout),
        logging.FileHandler('dockerfile_updater_api.log')
    ]
)
logger = logging.getLogger(__name__)

Classes and data classes

What was interesting was the job manager class that has been introduced. This is not something that I had thought about, however, in terms of scaling this tiny program into a more robust and larger service, this is definitely something that would be needed. Below are both the classes that were instantiated and the dataclass objects. All of the classes and data classes that have been added are good to have, and provide a clearer definition of the "what" each type of data is.

class JobStatus(str, Enum):
    """Job status enumeration."""
    PENDING = "pending"
    RUNNING = "running"
    COMPLETED = "completed"
    FAILED = "failed"


@dataclass
class Config:
    """Configuration class for the application."""
    api_version: str = "2022-11-28"
    timeout: int = 30
    max_retries: int = 3
    ai_model: str = "claude-opus-4-1-20250805"


class DockerfileUpdateRequest(BaseModel):
    """Request model for Dockerfile update."""
    owner: str
    repo: str
    branch: str = "main"
    github_token: str
    dockerfile_path: str = "Dockerfile"
    commit_message: Optional[str] = "Updated Dockerfile FROM via AI"
    dry_run: bool = False

    @field_validator('owner', 'repo')
    def validate_not_empty(cls, v):
        if not v or not v.strip():
            raise ValueError('Field cannot be empty')
        return v.strip()

    @field_validator('github_token')
    def validate_github_token(cls, v):
        if not v or len(v) < 10:
            raise ValueError('Invalid GitHub token')
        return v


class DockerfileUpdateResponse(BaseModel):
    """Response model for Dockerfile update."""
    job_id: str
    status: JobStatus
    message: str
    timestamp: datetime


class JobStatusResponse(BaseModel):
    """Response model for job status."""
    job_id: str
    status: JobStatus
    message: str
    timestamp: datetime
    result: Optional[Dict[str, Any]] = None
    error: Optional[str] = None


@dataclass
class JobResult:
    """Job result data class."""
    job_id: str
    status: JobStatus
    message: str
    timestamp: datetime
    result: Optional[Dict[str, Any]] = None
    error: Optional[str] = None


class DockerfileUpdaterError(Exception):
    """Custom exception for Dockerfile updater errors."""
    pass

HTTP Client class

The HTTP client class is client that that is used by the service to update the dockerfile. This is performed via a REST API call from the client to the github API. Claude has correctly added session handling, error handling and exception code.

class HTTPClient:
    """HTTP client with retry logic and proper error handling."""

    def __init__(self, timeout: int = 30, max_retries: int = 3):
        self.session = requests.Session()
        self.timeout = timeout

        # Configure retry strategy
        retry_strategy = Retry(
            total=max_retries,
            status_forcelist=[429, 500, 502, 503, 504],
            allowed_methods=["HEAD", "GET", "OPTIONS", "PUT"],
            backoff_factor=1
        )

        adapter = HTTPAdapter(max_retries=retry_strategy)
        self.session.mount("http://", adapter)
        self.session.mount("https://", adapter)

    def get(self, url: str, **kwargs) -> requests.Response:
        """Make GET request with error handling."""
        try:
            kwargs.setdefault('timeout', self.timeout)
            response = self.session.get(url, **kwargs)
            response.raise_for_status()
            return response
        except requests.exceptions.RequestException as e:
            logger.error(f"GET request failed for {url}: {e}")
            raise DockerfileUpdaterError(f"HTTP GET failed: {e}")

    def put(self, url: str, **kwargs) -> requests.Response:
        """Make PUT request with error handling."""
        try:
            kwargs.setdefault('timeout', self.timeout)
            response = self.session.put(url, **kwargs)
            response.raise_for_status()
            return response
        except requests.exceptions.RequestException as e:
            logger.error(f"PUT request failed for {url}: {e}")
            raise DockerfileUpdaterError(f"HTTP PUT failed: {e}")

GithubAPI Class

This class is specifically for dealing with the Github API. This is slightly different from the HTTP client that provides the transport.

There are four different methods that are to:

Get the repo contents
Get the file contents
Get the file SHA
Commit the file (with return and error codes)

All of this code is relatively robust, object oriented, and has appropriate error handling.

From where we started with a static procedural script, to get to here is quite amazing.

class GitHubAPI:
    """GitHub API client with proper error handling."""

    def __init__(self, token: str, owner: str, repo: str, config: Config, http_client: HTTPClient):
        self.token = token
        self.owner = owner
        self.repo = repo
        self.config = config
        self.http_client = http_client
        self.base_url = "https://api.github.com"
        self.headers = {
            "Authorization": f"Bearer {token}",
            "Accept": "application/vnd.github+json",
            "X-GitHub-Api-Version": config.api_version,
            "Content-Type": "application/json"
        }

    def get_repository_contents(self) -> List[Dict[str, Any]]:
        """Get repository contents."""
        url = f"{self.base_url}/repos/{self.owner}/{self.repo}/contents/"
        logger.info(f"Fetching repository contents from {url}")

        response = self.http_client.get(url, headers=self.headers)

        if response.status_code == 204:
            raise DockerfileUpdaterError("Repository is empty or deployment in progress")

        try:
            return response.json()
        except json.JSONDecodeError as e:
            logger.error(f"Failed to parse JSON response: {e}")
            raise DockerfileUpdaterError("Invalid JSON response from GitHub API")

    def get_file_content(self, download_url: str) -> str:
        """Download file content from GitHub."""
        logger.info(f"Downloading file from {download_url}")
        response = self.http_client.get(download_url)
        return response.text

    def get_file_sha(self, file_path: str, branch: str) -> Optional[str]:
        """Get the SHA of an existing file."""
        url = f"{self.base_url}/repos/{self.owner}/{self.repo}/contents/{file_path}"
        params = {"ref": branch}

        try:
            response = self.http_client.get(url, headers=self.headers, params=params)
            return response.json()["sha"]
        except DockerfileUpdaterError:
            # File doesn't exist
            return None

    def commit_file(self, file_path: str, content: str, commit_message: str, branch: str) -> Dict[str, Any]:
        """Commit file to repository."""
        url = f"{self.base_url}/repos/{self.owner}/{self.repo}/contents/{file_path}"

        # Encode content to base64
        try:
            content_encoded = base64.b64encode(content.encode('utf-8')).decode('utf-8')
        except UnicodeEncodeError as e:
            raise DockerfileUpdaterError(f"Failed to encode file content: {e}")

        payload = {
            "message": commit_message,
            "content": content_encoded,
            "branch": branch
        }

        # Get existing file SHA if file exists
        existing_sha = self.get_file_sha(file_path, branch)
        if existing_sha:
            payload["sha"] = existing_sha
            logger.info(f"Updating existing file {file_path}")
        else:
            logger.info(f"Creating new file {file_path}")

        response = self.http_client.put(url, headers=self.headers, data=json.dumps(payload))
        return response.json()

AI Class

This class is the class that makes the called to the AI model. There are two methods here:

Find the dockerfile url in the repo
Update the dockerfile base image

The two interesting things here are that clause has not altered the prompts, json schemas or changed the use the langchain.

Claude has continued to provide robust error handling and has turned my code into a class.

class AIAnalyzer:
    """AI-powered Dockerfile analyzer."""

    def __init__(self, model_name: str = "claude-opus-4-1-20250805"):
        try:
            # Ensure Anthropic API key is available
            if not os.environ.get("ANTHROPIC_API_KEY"):
                raise DockerfileUpdaterError("ANTHROPIC_API_KEY environment variable is required")

            self.model = init_chat_model(model_name, model_provider="anthropic")
            logger.info(f"Initialized AI model: {model_name}")
        except Exception as e:
            logger.error(f"Failed to initialize AI model: {e}")
            raise DockerfileUpdaterError(f"AI model initialization failed: {e}")

    def find_dockerfile_url(self, repository_contents: List[Dict[str, Any]], dockerfile_path: str) -> str:
        """Find Dockerfile URL in repository contents using AI."""
        # First try to find the file directly by name
        for item in repository_contents:
            if item.get("name") == dockerfile_path and item.get("type") == "file":
                return item.get("download_url")

        # If not found directly, use AI to search
        dockerfile_schema = {
            "title": "dockerfile",
            "description": "Format of dockerfile links",
            "type": "object",
            "properties": {
                "textresponse": {
                    "type": "string",
                    "description": "The text response portion",
                },
                "fileurl": {
                    "type": "string",
                    "description": "The actual url of the file",
                },
            },
            "required": ["textresponse", "fileurl"],
        }

        structured_model = self.model.with_structured_output(dockerfile_schema)

        try:
            response = structured_model.invoke(
                f'Find dockerfile named "{dockerfile_path}" in array {repository_contents} return only value do
wnload_url'
            )
            logger.info("Successfully found Dockerfile URL using AI")
            return response["fileurl"]
        except Exception as e:
            logger.error(f"AI analysis failed for finding Dockerfile: {e}")
            raise DockerfileUpdaterError(f"Failed to find Dockerfile '{dockerfile_path}': {e}")

    def update_dockerfile_base_image(self, dockerfile_content: str) -> str:
        """Update Dockerfile base image using AI."""
        dockerfile_schema = {
            "title": "dockerfile",
            "description": "the dockerfile",
            "type": "object",
            "properties": {
                "textresponse": {
                    "type": "string",
                    "description": "The text response portion",
                },
                "dockerfile": {
                    "type": "string",
                    "description": "the dockerfile",
                },
            },
            "required": ["textresponse", "dockerfile"],
        }

        structured_model = self.model.with_structured_output(dockerfile_schema)

        try:
            prompt = (
                f'Update the FROM command to be the latest baseimage version for {dockerfile_content}, '
                'return the updated dockerfile. Make no changes if the baseimage is already at the latest versi
on'
            )
            response = structured_model.invoke(prompt)
            logger.info("Successfully analyzed Dockerfile for base image updates")
            return response["dockerfile"]
        except Exception as e:
            logger.error(f"AI analysis failed for updating Dockerfile: {e}")
            raise DockerfileUpdaterError(f"Failed to update Dockerfile: {e}")

Job Manager class

This is the one thing that I just didn't even think of. Given that I have an API calling out to another API, and this is likely to be asynchronous, job handling is required. Again, the methods here are impressive.

Create job
Update job
Get job
List all jobs

This includes having multiple status types for jobs, including PENDING and COMPLETED.

class JobManager:
    """Manages background jobs."""

    def __init__(self):
        self.jobs: Dict[str, JobResult] = {}

    def create_job(self, job_id: str) -> None:
        """Create a new job."""
        self.jobs[job_id] = JobResult(
            job_id=job_id,
            status=JobStatus.PENDING,
            message="Job created",
            timestamp=datetime.now()
        )

    def update_job(self, job_id: str, status: JobStatus, message: str,
                   result: Optional[Dict[str, Any]] = None, error: Optional[str] = None) -> None:
        """Update job status."""
        if job_id in self.jobs:
            self.jobs[job_id].status = status
            self.jobs[job_id].message = message
            self.jobs[job_id].timestamp = datetime.now()
            self.jobs[job_id].result = result
            self.jobs[job_id].error = error

    def get_job(self, job_id: str) -> Optional[JobResult]:
        """Get job by ID."""
        return self.jobs.get(job_id)

    def list_jobs(self) -> List[JobResult]:
        """List all jobs."""
        return list(self.jobs.values())

Dockerfile Update Service

This is a wrapper class that pulls together using the HTTP client above, and the AI analyser class to make the changes to the dockerfile. This class also performs implementation of the job handling code.

class DockerfileUpdaterService:
    """Main service class."""

    def __init__(self, config: Config):
        self.config = config
        self.http_client = HTTPClient(config.timeout, config.max_retries)
        self.ai_analyzer = AIAnalyzer(config.ai_model)
        self.job_manager = JobManager()

    async def update_dockerfile(self, job_id: str, request: DockerfileUpdateRequest) -> None:
        """Update Dockerfile in background."""
        try:
            self.job_manager.update_job(job_id, JobStatus.RUNNING, "Starting Dockerfile update process")

            # Initialize GitHub API client
            github_api = GitHubAPI(
                request.github_token,
                request.owner,
                request.repo,
                self.config,
                self.http_client
            )

            # Get repository contents
            contents = github_api.get_repository_contents()
            self.job_manager.update_job(job_id, JobStatus.RUNNING, "Retrieved repository contents")

            # Find Dockerfile URL
            dockerfile_url = self.ai_analyzer.find_dockerfile_url(contents, request.dockerfile_path)
            self.job_manager.update_job(job_id, JobStatus.RUNNING, f"Found Dockerfile at: {dockerfile_url}")

            # Download current Dockerfile
            current_dockerfile = github_api.get_file_content(dockerfile_url)
            current_first_line = current_dockerfile.split('\n', 1)[0] if current_dockerfile else ""

            # Update Dockerfile using AI
            updated_dockerfile = self.ai_analyzer.update_dockerfile_base_image(current_dockerfile)
            updated_first_line = updated_dockerfile.split('\n', 1)[0] if updated_dockerfile else ""

            # Check if changes are needed
            if current_first_line == updated_first_line:
                self.job_manager.update_job(
                    job_id,
                    JobStatus.COMPLETED,
                    "No changes needed - Dockerfile is already up to date",
                    result={
                        "changed": False,
                        "current_from": current_first_line,
                        "updated_from": updated_first_line
                    }
                )
                return

            result_data = {
                "changed": True,
                "current_from": current_first_line,
                "updated_from": updated_first_line,
                "dry_run": request.dry_run
            }

            # Commit updated Dockerfile (unless dry run)
            if not request.dry_run:
                commit_result = github_api.commit_file(
                    file_path=request.dockerfile_path,
                    content=updated_dockerfile,
                    commit_message=request.commit_message,
                    branch=request.branch
                )

                result_data.update({
                    "commit_sha": commit_result['commit']['sha'],
                    "file_url": commit_result['content']['html_url']
                })

                self.job_manager.update_job(
                    job_id,
                    JobStatus.COMPLETED,
                    "Successfully updated and committed Dockerfile",
                    result=result_data
                )
            else:
                self.job_manager.update_job(
                    job_id,
                    JobStatus.COMPLETED,
                    "Dry run completed - changes detected but not committed",
                    result=result_data
                )

        except Exception as e:
            error_message = str(e)
            logger.error(f"Job {job_id} failed: {error_message}")
            self.job_manager.update_job(
                job_id,
                JobStatus.FAILED,
                "Job failed with error",
                error=error_message
            )

API

This portion of the code implements the API and reverse proxy / middleware layer of my API. This uses the config class and passes that to the updater service.

# Initialize FastAPI app
app = FastAPI(
    title="Dockerfile Updater API",
    description="REST API for updating Dockerfile base images using AI",
    version="1.0.0"
)

# Add CORS middleware
app.add_middleware(
    CORSMiddleware,
    allow_origins=["*"],
    allow_credentials=True,
    allow_methods=["*"],
    allow_headers=["*"],
)

# Initialize services
config = Config()
service = DockerfileUpdaterService(config)

# Security
security = HTTPBearer(auto_error=False)


async def get_api_key(credentials: HTTPAuthorizationCredentials = Depends(security)):
    """Validate API key (optional)."""
    # You can implement API key validation here if needed
    return credentials

Routes

The code below implements a number of routes. These are:

/health GET
/update-dockerfile POST
/jobs GET
/jobs/{job} GET
/jobs/{job} DELETE

Each of these routes can be called and with the appropriate headers is the primary way to interact with the service.

@app.get("/health")
async def health_check():
    """Health check endpoint."""
    return {"status": "healthy", "timestamp": datetime.now()}


@app.post("/update-dockerfile", response_model=DockerfileUpdateResponse)
async def update_dockerfile(
    request: DockerfileUpdateRequest,
    background_tasks: BackgroundTasks,
    credentials: HTTPAuthorizationCredentials = Depends(get_api_key)
):
    """Update Dockerfile base image."""
    try:
        # Generate job ID
        job_id = str(uuid.uuid4())

        # Create job
        service.job_manager.create_job(job_id)

        # Start background task
        background_tasks.add_task(service.update_dockerfile, job_id, request)

        return DockerfileUpdateResponse(
            job_id=job_id,
            status=JobStatus.PENDING,
            message="Job created and queued for processing",
            timestamp=datetime.now()
        )

    except Exception as e:
        logger.error(f"Failed to create update job: {e}")
        raise HTTPException(status_code=500, detail=str(e))


@app.get("/jobs/{job_id}", response_model=JobStatusResponse)
async def get_job_status(job_id: str):
    """Get job status by ID."""
    job = service.job_manager.get_job(job_id)

    if not job:
        raise HTTPException(status_code=404, detail="Job not found")

    return JobStatusResponse(**asdict(job))


@app.get("/jobs", response_model=List[JobStatusResponse])
async def list_jobs():
    """List all jobs."""
    jobs = service.job_manager.list_jobs()
    return [JobStatusResponse(**asdict(job)) for job in jobs]


@app.delete("/jobs/{job_id}")
async def delete_job(job_id: str):
    """Delete job by ID."""
    if job_id not in service.job_manager.jobs:
        raise HTTPException(status_code=404, detail="Job not found")

    del service.job_manager.jobs[job_id]
    return {"message": "Job deleted successfully"}

Main function

The main function is very standard. It starts a uvicorn server on port 8000 that hosts my code.

if __name__ == "__main__":
    import uvicorn

    # Ensure required environment variables
    if not os.environ.get("ANTHROPIC_API_KEY"):
        logger.error("ANTHROPIC_API_KEY environment variable is required")
        sys.exit(1)

    uvicorn.run(app, host="0.0.0.0", port=8000, log_level="info")

Conclusion

This has been a long journey so far for the reader, but for me as a developer it's been very quick. It took me only a few minutes to refactor the code into a service using Claude. I have a robust service now that allows me to get Dockerfiles updated without too much difficulty. I just need to feed in the right paramters and away it goes, and performs the checks on my behalf, does the analysis, and updates the repository.

This is a neat way to keep your Dockerfiles updated and your repos clean!

Github dockerfile service using AI - Part 1

codecowboydotio — Tue, 25 Nov 2025 10:26:51 +0000

Intro

I have been fooling around a lot with ai recently, and I thought I would write something about what I've been doing. There are a few things that I've been doing, and they're all fascinating.

Writing average looking code using minimum viable prompts.
Getting this average looking code refactored to be a robust service.
Generating API documentation - and having Claude fix this code.

This is part one of a small series that I have created to walk through the process I went through to get decent code.

A crazy idea - update my dockerfiles

Most Dockerfiles have a fixed base image line that looks something like this:

FROM docker.io/library/ubuntu:20.04

This is painful to trawl through and update. Not least because I don't actually know what the latest version is, and I'm not particularly keen on just using the latest tag. In addition to this, depending on the version of OS that I'm using (Alpine, Ubuntu etc) I don't want to have to keep on top of every single image and tag.

I wanted something that would do this for me.

This seemed like the perfect use for an LLM and the ability to get the LLM to simply tell me what the latest version for a given base image is.

Version 1 - the make it work code

My very first version of code, was hand written and was very procedural. This is the "make it work" code, where I experimented with several different ways of making calls to the LLM and github.

This is not the full code base, but rather the relevant parts for version one.

Initially, I make a REST call to the github API to get the file contents of my github repository. Note that the repo is static in this version.

The appropriate headers are passed, and the appropriate token. After performing a GET request, the answer is a list of files from the repository.

url='https://api.github.com/repos/codecowboydotio/swapi-json-server/contents/'

headers = {
  "Accept": "application/vnd.github+json",
  "X-GitHub-Api-Version": "2022-11-28",
  "Authorization": f'Bearer {GIT_TOKEN}'
}

query_answer = requests.get(url, headers=headers)
query_response_code = query_answer.status_code
if query_response_code  == 204:
  print("Please check github actions for the status of the deployment")
else:
  print("Contacting git....")
  print("Got response...")

The response from the code above is a JSON response of files that looks something like this:

[{"name":".file","path":".file","sha":"XXX","size":0,"url":"https://api.github.com/repos/codecowboydotio/swapi-json-server/contents/.file?ref=main","html_url":"https://github.com/codecowboydotio/swapi-json-server/tree/main/.file","git_url":"https://api.github.com/repos/codecowboydotio/swapi-json-server/git/trees/XXX","download_url":null,"type":"dir","_links":{"self":"https://api.github.com/repos/codecowboydotio/swapi-json-server/contents/.file?ref=main","git":"https://api.github.com/repos/codecowboydotio/swapi-json-server/git/trees/XXX","html":"https://github.com/codecowboydotio/swapi-json-server/tree/main/.file"}}

The JSON blob contains all of the information that I need to determine if my file is a Dockerfile, and the location of that file.

Enter the first LLM call

What happens next is that I use Langchain to instantiate the claude opus 4.1 model. The reason for langchain here is that I have been experimenting with Langchain and how it fits into my development processes. It does provide an interesting way of allowing for structured output - which allows a model to return data in a predictable way.

https://docs.langchain.com/oss/python/langchain/structured-output

from langchain.chat_models import init_chat_model

model = init_chat_model("claude-opus-4-1-20250805", model_provider="anthropic")

dockerfiles_json_schema = {
    "title": "dockerfile",
    "description": "Format of dockerfile links",
    "type": "object",
    "properties": {
        "textresponse": {
            "type": "string",
            "description": "The text response portion",
        },
        "fileurl": {
            "type": "string",
            "description": "The actual url of the file",
        },
    },
    "required": ["textresponse", "fileurl"],
}

structured_model = model.with_structured_output(dockerfiles_json_schema)
print("Initializing model...")
try:
  response = structured_model.invoke(f'find dockerfiles in array {query_answer.text} return only value download_url')
except Exception as e:
  print(f"LLM error: {e}")
  exit(1)
print("Found dockerfile...")

Structured output allows me to define a JSON schema of what I would like the output to look like. This allows me to pass a schema of what I would like the output to look like, and than when I instanitate the model, reference the schema.

I essentially split the structured output into "textresponse" or junk, and "fileurl" which is the information I really want.

Finally, I use a prompt that invokes the model using my structured output schema.

structured_model = model.with_structured_output(dockerfiles_json_schema)
print("Initializing model...")
try:
  response = structured_model.invoke(f'find dockerfiles in array {query_answer.text} return only value download_url')

One important thing to note here is that I have used a "cut down prompt". Not content with learning one thing at a time, I decided to also experiment with smaller prompts, which reduces the context window and tokens that I use. The simple prompt:

find dockerfiles in array {query_answer.text} return only value download_url

Is missing the word the among some other syntactic values if you were asking a person the same thing. The meaning is still 100% clear, but I have saved on some tokens. There is another blog post on this coming soon.

The output from this section if raw output and debug were enabled would look like this:

Found dockerfile...
{'textresponse': 'Found 1 Dockerfile in the repository:', 'fileurl': 'https://raw.githubusercontent.com/codecowboydotio/swapi-json-server/main/Dockerfile'}

This contains the fileurl and the textresponse as items that I can call as part of the response.

Getting the Dockerfile

The next piece of the code takes the dockerfile url, performs an HTTP GET on the file, and isolates the first line. The purpose here is to check the FROM command and again, feed this to an LLM via another Langchain structured reponse and a schema. This will allow me to ask the LLM to find the latest image tag.


print("Pulling dockerfile from git....")
dockerfile = requests.get(response["fileurl"])
docker_response = dockerfile.text
first_line = docker_response.split('\n', 1)[0]
print("Original: " + first_line)

dockerfile_json_schema = {
    "title": "dockerfile",
    "description": "the dockerfile",
    "type": "object",
    "properties": {
        "textresponse": {
            "type": "string",
            "description": "The text response portion",
        },
        "dockerfile": {
            "type": "string",
            "description": "the dockerfile",
        },
    },
    "required": ["textresponse", "dockerfile"],
}
print("Sending entire dockerfile to LLM to determine latest baseimage...")
dockerfile_structured_model = model.with_structured_output(dockerfile_json_schema)
response = dockerfile_structured_model.invoke(f'Update the FROM command to be the latest baseimage version for {dockerfile.text}, return the updated dockerfile make no changes if the baseimage is already at the latest version')

The thing to note here is that I am explicitly asking that no changes should be made if the baseimage is already at the latest version. This is to ensure that nothing odd like the addition of the latest tag get added.

The other thing this prompt does is to explicitly ask to update the FROM command, and no other part of the Dockerfile.

The response below is the entire Dockerfile, not just the FROM line. This means that the entire Dockerfile is sent back. An if statement checks to make sure that the FROM line is not the same. If it is, no further action is taken. If it is not, then I call a class to commit the Dockerfile to github.


llm_docker_response = response["dockerfile"]
llm_first_line = llm_docker_response.split('\n', 1)[0]
print("Replacement: " + llm_first_line)

if (llm_first_line == first_line):
  print("Original and replacement are the same..... doing nothing")
  exit(1)
else:
  # File details
  file_path = "Dockerfile"
  file_content = response["dockerfile"]

  commit_message = "Updated Dockerfile FROM via AI"
  branch = "main"  # or "master" depending on your default branch

  try:
      # Create committer instance
      committer = GitHubCommitter(GITHUB_TOKEN, OWNER, REPO)

Let's run it

The proof is in the pudding here. A run of program where no changes are to be made looks like this:

./git-test.py
Contacting git....
Got response...
Initializing model...
Found dockerfile...
Pulling dockerfile from git....
Original: FROM docker.io/library/ubuntu:24.04
Sending entire dockerfile to LLM to determine latest baseimage...
Replacement: FROM docker.io/library/ubuntu:24.04
Original and replacement are the same..... doing nothing

In this case, the original and latest are at the same version. Therefore there are no changes to be made.

When I update the FROM command to be an older version of the dockerfile baseimage, then the replacement is found, and the file is automatically committed to git.

Contacting git....
Got response...
Initializing model...
Found dockerfile...
Pulling dockerfile git....
Original: FROM docker.io/library/ubuntu:20.04
Sending entire dockerfile to LLM to determine latest baseimage...
Replacement: FROM docker.io/library/ubuntu:24.04
Committing file 'Dockerfile' to codecowboydotio/swapi-json-server...
✅ Success!
Commit SHA: XXX
File URL: https://github.com/codecowboydotio/swapi-json-server/blob/main/Dockerfile

The actual commit in github looks like this. You can see the commit message that says it was updated by AI.

Github code

Once I had my very very basic working code to give me an update by calling an LLM, I asked claude to generate some python code to perform a github update, and just pasted it into my codebase.

It worked straight away.

#!/usr/bin/python

import getpass
import os
import requests
import json
from langchain.chat_models import init_chat_model
import base64
from typing import Optional


if not os.environ.get("GIT_PAT_AI"):
  os.environ["GIT_PAT_AI"] = getpass.getpass("Enter API key for Github: ")
else:
  GIT_TOKEN=os.environ["GIT_PAT_AI"]

if not os.environ.get("ANTHROPIC_API_KEY"):
  os.environ["ANTHROPIC_API_KEY"] = getpass.getpass("Enter API key for Anthropic: ")


GITHUB_TOKEN = GIT_TOKEN # Create at https://github.com/settings/tokens
OWNER = "codecowboydotio"  # Your GitHub username or organization
REPO = "swapi-json-server"  # Repository name

class GitHubCommitter:
    def __init__(self, token: str, owner: str, repo: str):
        """
        Initialize GitHub committer.

        Args:
            token: GitHub personal access token
            owner: Repository owner (username or organization)
            repo: Repository name
        """
        self.token = token
        self.owner = owner
        self.repo = repo
        self.base_url = "https://api.github.com"
        self.headers = {
            "Authorization": f"token {token}",
            "Accept": "application/vnd.github.v3+json",
            "Content-Type": "application/json"
        }

    def get_file_sha(self, file_path: str, branch: str = "main") -> Optional[str]:
        """Get the SHA of an existing file (needed for updates)."""
        url = f"{self.base_url}/repos/{self.owner}/{self.repo}/contents/{file_path}"
        params = {"ref": branch}

        response = requests.get(url, headers=self.headers, params=params)

        if response.status_code == 200:
            return response.json()["sha"]
        elif response.status_code == 404:
            return None  # File doesn't exist
        else:
            response.raise_for_status()

    def commit_file(self, file_path: str, content: str, commit_message: str,
                   branch: str = "main", update_existing: bool = True) -> dict:
        """
        Create or update a file and commit it.

        Args:
            file_path: Path to the file in the repository
            content: File content as string
            commit_message: Commit message
            branch: Branch to commit to
            update_existing: Whether to update if file exists

        Returns:
            API response as dictionary
        """
        url = f"{self.base_url}/repos/{self.owner}/{self.repo}/contents/{file_path}"

        # Encode content to base64
        content_encoded = base64.b64encode(content.encode('utf-8')).decode('utf-8')

        # Prepare the payload
        payload = {
            "message": commit_message,
            "content": content_encoded,
            "branch": branch
        }

        # Check if file exists and get its SHA if it does
        if update_existing:
            existing_sha = self.get_file_sha(file_path, branch)
            if existing_sha:
                payload["sha"] = existing_sha

        # Make the request
        response = requests.put(url, headers=self.headers, data=json.dumps(payload))

        if response.status_code in [200, 201]:
            return response.json()
        else:
            response.raise_for_status()



url='https://api.github.com/repos/codecowboydotio/swapi-json-server/contents/'

headers = {
  "Accept": "application/vnd.github+json",
  "X-GitHub-Api-Version": "2022-11-28",
  "Authorization": f'Bearer {GIT_TOKEN}'
}

query_answer = requests.get(url, headers=headers)
query_response_code = query_answer.status_code
if query_response_code  == 204:
  print("Please check github actions for the status of the deployment")
else:
  print("Contacting git....")
  print("Got response...")
 # print(query_answer.text)


model = init_chat_model("claude-opus-4-1-20250805", model_provider="anthropic")
#model = init_chat_model("claude-3-5-sonnet-latest", model_provider="anthropic")

dockerfiles_json_schema = {
    "title": "dockerfile",
    "description": "Format of dockerfile links",
    "type": "object",
    "properties": {
        "textresponse": {
            "type": "string",
            "description": "The text response portion",
        },
        "fileurl": {
            "type": "string",
            "description": "The actual url of the file",
        },
    },
    "required": ["textresponse", "fileurl"],
}

structured_model = model.with_structured_output(dockerfiles_json_schema)
print("Initializing model...")
try:
  response = structured_model.invoke(f'find dockerfiles in array {query_answer.text} return only value download_url')
except Exception as e:
  print(f"LLM error: {e}")
  exit(1)
print("Found dockerfile...")
#print(response)

print(f"Pulling dockerfile {response["fileurl"]} from git....")
dockerfile = requests.get(response["fileurl"])
docker_response = dockerfile.text
first_line = docker_response.split('\n', 1)[0]
print("Original: " + first_line)

dockerfile_json_schema = {
    "title": "dockerfile",
    "description": "the dockerfile",
    "type": "object",
    "properties": {
        "textresponse": {
            "type": "string",
            "description": "The text response portion",
        },
        "dockerfile": {
            "type": "string",
            "description": "the dockerfile",
        },
    },
    "required": ["textresponse", "dockerfile"],
}
print("Sending entire dockerfile to LLM to determine latest baseimage...")
dockerfile_structured_model = model.with_structured_output(dockerfile_json_schema)
#response = dockerfile_structured_model.invoke(f'Update the FROM command to be the latest baseimage version for {dockerfile.text}, return the updated dockerfile')
response = dockerfile_structured_model.invoke(f'Update the FROM command to be the latest baseimage version for {dockerfile.text}, return the updated dockerfile make no changes if the baseimage is already at the latest version')
#print("===========")
#print(response["dockerfile"])


llm_docker_response = response["dockerfile"]
llm_first_line = llm_docker_response.split('\n', 1)[0]
print("Replacement: " + llm_first_line)

if (llm_first_line == first_line):
  print("Original and replacement are the same..... doing nothing")
  exit(1)
else:
  # File details
  file_path = "Dockerfile"
  file_content = response["dockerfile"]
  #file_content = """Hello, World!
  #This is a test file created via GitHub API.
  #Current timestamp: """ + str(requests.get("http://worldtimeapi.org/api/timezone/Australia/Melbourne").json().get("datetime", "unknown"))

  commit_message = "Updated Dockerfile FROM via AI"
  branch = "main"  # or "master" depending on your default branch

  try:
      # Create committer instance
      committer = GitHubCommitter(GITHUB_TOKEN, OWNER, REPO)

      # Commit the file
      print(f"Committing file '{file_path}' to {OWNER}/{REPO}...")
      result = committer.commit_file(
          file_path=file_path,
          content=file_content,
          commit_message=commit_message,
          branch=branch
      )

      print("✅ Success!")
      print(f"Commit SHA: {result['commit']['sha']}")
      print(f"File URL: {result['content']['html_url']}")
  except requests.exceptions.HTTPError as e:
      print(f"❌ HTTP Error: {e}")
      print(f"Response: {e.response.text}")
  except Exception as e:
      print(f"❌ Error: {e}")

Conclusion

This is still really really nasty code. It works, but it's a single shot procedural nightmare. What comes next is even more exciting where I turn this into a complete working service, that can be run on any repository. The way that I do this is even more fascinating than the original exploration.

Pulumi - DNS in AWS

codecowboydotio — Mon, 24 Nov 2025 08:19:01 +0000

Pulumi - DNS in AWS

What

This post explores two things. First it explores setting up a DNS domain, zone and records as part of route 53. Then, we will go further and explore using output of other services as records. For the purposes of demonstrtation this will be a new record that will be created if an EC2 instance exists.

Why

The pattern of route53 is the last step in the infrastructure journey. Setting up infrastructure, and assigning a friendly domain name, and www record to the infrastructure is a relatively normal thing to do.

The pulumi templates in this article will allow you to deploy this pattern quickly and easily.

What

I will re-use the EC2 instance that I have in another Pulumi stack in order to have a virtual machine up and running.

I will also delegate a domain to AWS at my hosting provider. Once done, I will show how to use both the EC2 IP address and create a www record. I will also show how to bypass doing this and create a default record.

Configuration

The configuration of the template is simple and standard. Import the relevant modules, and set the config imports to get the variables from the stack config.

Stack reference

There is a special variable called other_stack in my pulumi config file. This is used in my code as a reference to import a second stack. The second stack is the virtual machine stack. This is where I will get the IP address for the www record from if this variable is defined.

The variable is used as a boolean to determine if a stackreference is used or not. A stack reference is a reference to another stack that allows you to use the outputs of another stack in the current program.

The code snippet below has a standard python if statement that is run if the variable "other_stack" is true. If true, we will import the stack name and import as a stack reference. Once imported, we will then use the IP address of the EC2 instance created in the other stack, and assign it to a variable.

import pulumi
import pulumi_aws as aws

config = pulumi.Config()
project_name = config.get("project_nane", "svk")
project_purpose = config.get("purpose", "route53")
zone_name = config.get("zone_name", "example.com")
use_other_stack = config.get_bool("other_stack", False)

if use_other_stack == True:
  source_stack_name = config.get("sourceStackName")
  source_stack = pulumi.StackReference(source_stack_name)
  vm_ip = source_stack.get_output("public_ip")

The stack reference here is a way of referencing another pulumi stack, and its output variables. This allows me to use the IP address of an EC2 instance in a completely different stack as part of my DNS zone.

DNS zones

The code below configures two DNS zones. These are traditional DNS zones. One is a "main" zone, and one is a sub zone.

The main zone definition is relatively simple. Once created, it returns the nameservers that AWS assigns to the zone. These can be used for redelegation of the zone to the AWS nameservers.

mainzone = aws.route53.Zone("mainzone",
    name = zone_name,
)

dev = aws.route53.Zone("dev",
    name="dev." + zone_name,
    tags={
        "Environment": "dev",
    })
dev_ns = aws.route53.Record("dev-ns",
    zone_id=mainzone.zone_id,
    name="dev." + zone_name,
    type=aws.route53.RecordType.NS,
    ttl=30,
    records=dev.name_servers)

Note the the code above also creates a sub zone. This is named dev. In all of my examples I am using one of my test zones, svkcode.com. This means that the sub zone will be named dev.svkcode.com

Once the zone has been created it looks like this in AWS:

DNS Records

The code below creates two DNS records. One for each zone. I have created a www record. This means that there will be two records

www.svkcode.com
www.dev.svkcode.com

The thing to note is that I am using a standard python if statement on one of the records. This means that the record will be conditionally created if we are using an external stack.

If the external stack variable is set, the IP address that is used is the IP address of the virtual machine from our external stack.

This is to cater for times when I do not have an external address from outside this stack that I want to use. The neat thing here is that it is a standard python if statement.

if use_other_stack == True:
  www = aws.route53.Record("www",
      zone_id=mainzone.zone_id,
      name="www." + zone_name,
      type=aws.route53.RecordType.A,
      ttl=300,
      #records=["1.2.3.4"]
      records=[vm_ip]
  )
else:
  www = aws.route53.Record("www",
      zone_id=mainzone.zone_id,
      name="www." + zone_name,
      type=aws.route53.RecordType.A,
      ttl=300,
      records=["1.2.3.4"]
  )

devwww = aws.route53.Record("devwww",
    zone_id=dev.zone_id,
    name="www.dev." + zone_name,
    type=aws.route53.RecordType.A,
    ttl=300,
    records=["1.1.1.1"]
)

When the zones and records have been created they look like the image below in the console.

Initial zone creation

On initial stack creation, we can see that two zones are created and the associated nameservers for each zone are output to the screen. These can be used for redelegation of the zone to AWS nameservers.

Updating (dev)

View in Browser (Ctrl+O): https://app.pulumi.com/XXX

     Type                        Name             Status
 +   pulumi:pulumi:Stack         aws-route53-dev  created (80s)
 +   ├─ aws:route53:Zone         mainzone         created (38s)
 +   ├─ aws:route53:Zone         dev              created (39s)
 +   ├─ aws:route53:Record       www              created (35s)
 +   ├─ aws:route53:Record       devwww           created (35s)
 +   ├─ aws:route53:Record       dev-ns           created (35s)
 +   └─ aws:route53:HealthCheck  healthcheck      created (2s)

Outputs:
    Dev Nameservers : [
        [0]: "ns-1016.awsdns-63.net"
        [1]: "ns-1231.awsdns-25.org"
        [2]: "ns-1562.awsdns-03.co.uk"
        [3]: "ns-378.awsdns-47.com"
    ]
    Main Nameservers: [
        [0]: "ns-1272.awsdns-31.org"
        [1]: "ns-1706.awsdns-21.co.uk"
        [2]: "ns-185.awsdns-23.com"
        [3]: "ns-804.awsdns-36.net"
    ]
    stack flag      : false

Resources:
    + 7 created

Duration: 1m21s

Once the redelegation at the domain registrar is complete, we can see that the zone svkcode.org is correctly delegated to the AWS nameservers. The nameservers from my dig command match those that were output at the time of stack creation.

dig NS svkcode.com

; <<>> DiG 9.18.33 <<>> NS svkcode.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 28036
;; flags: qr rd ra; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 65494
;; QUESTION SECTION:
;svkcode.com.                   IN      NS

;; ANSWER SECTION:
svkcode.com.            21600   IN      NS      ns-1272.awsdns-31.org.
svkcode.com.            21600   IN      NS      ns-185.awsdns-23.com.
svkcode.com.            21600   IN      NS      ns-1706.awsdns-21.co.uk.
svkcode.com.            21600   IN      NS      ns-804.awsdns-36.net.

;; Query time: 142 msec
;; SERVER: 127.0.0.53#53(127.0.0.53) (UDP)
;; WHEN: Sat Nov 15 18:12:19 AEDT 2025
;; MSG SIZE  rcvd: 177

Finally, when we use nslookup to test the records that were created we can see that the answer is 1.2.3.4. This means that the if statement has worked correctly and that we have elected not to use an external stack as the source for the www record.

nslookup www.svkcode.com

Server:         127.0.0.53
Address:        127.0.0.53#53

Non-authoritative answer:
Name:   www.svkcode.com
Address: 1.2.3.4

We can further validate that this is the expected behaviour by checking the stack configuration.

pulumi config

pulumi config setKEY              VALUE
aws:region       ap-southeast-2
other_stack      false
sourceStackName  codecowboydotio/aws-vm/dev
zone_name        svkcode.com
pulumi:tags      {"pulumi:template":"aws-python"}

We can see that the other_stack configuration is set to false. This means that the if statement in our code will fall through to the else statement and set a default record value of 1.2.3.4

Zone creation using other stack

When I change the stack configuration and set the other_stack variable to true, this should trigger creating a record using the IP address of an existing virtual machine from another stack.

The first thing I will do is create a virtual machine using my other stack.

Running the other stack, I get an output of the IP address of the virtual machne.

Outputs:
    public_dns: [unknown]
    public_ip : [unknown]

Resources:
    + 8 to create

Updating (dev)

View in Browser (Ctrl+O): https://app.pulumi.com/XXX

     Type                              Name            Status
 +   pulumi:pulumi:Stack               aws-vm-dev      created (32s)
 +   ├─ aws:ec2:Vpc                    svk-vm-vpc      created (1s)
 +   ├─ aws:ec2:SecurityGroup          svk-vm-sg       created (3s)
 +   ├─ aws:ec2:InternetGateway        svk-vm-vpc-igw  created (1s)
 +   ├─ aws:ec2:Subnet                 svk-vm-subnet   created (11s)
 +   ├─ aws:ec2:RouteTable             svk-vm-vpc-rt   created (1s)
 +   ├─ aws:ec2:RouteTableAssociation  svk-vm-vpc-rt   created (1s)
 +   └─ aws:ec2:Instance               svk-vm-server   created (13s)

Outputs:
    public_ip : "3.26.192.12"

Resources:
    + 8 created

Duration: 35s

Next, I change the configuration of the other_stack variable to true. This effectively tells my code to use the IP address of the virtual machine that I just created for the record www.svkcode.com

pulumi config set other_stack true

pulumi config

KEY              VALUE
aws:region       ap-southeast-2
other_stack      true
sourceStackName  codecowboydotio/aws-vm/dev
zone_name        svkcode.com
pulumi:tags      {"pulumi:template":"aws-python"}

With the value now set to true, I can run the stack to create my DNS zones again. As this has already been run, only an update will be performed. The update impacts only the www record.



Updating (dev)

View in Browser (Ctrl+O): https://app.pulumi.com/XXX

     Type                   Name             Status            Info
     pulumi:pulumi:Stack    aws-route53-dev
 ~   └─ aws:route53:Record  www              updated (40s)     [diff: ~records]

Outputs:
    Dev Nameservers : [
        [0]: "ns-1016.awsdns-63.net"
        [1]: "ns-1231.awsdns-25.org"
        [2]: "ns-1562.awsdns-03.co.uk"
        [3]: "ns-378.awsdns-47.com"
    ]
    Main Nameservers: [
        [0]: "ns-1272.awsdns-31.org"
        [1]: "ns-1706.awsdns-21.co.uk"
        [2]: "ns-185.awsdns-23.com"
        [3]: "ns-804.awsdns-36.net"
    ]
  ~ stack flag      : false => true

Resources:
    ~ 1 updated
    6 unchanged

Duration: 50s

When I check the DNS configuration thi time by querying the records, I can see that the record has been updated to the IP address of the virtual machine from my other stack.


Bash
nslookup www.svkcode.com

Server:         127.0.0.53
Address:        127.0.0.53#53

Non-authoritative answer:
Name:   www.svkcode.com
Address: 3.26.192.12

Finally, if I hit the resource with a web browser, I should see the pacman game. This is because the stack that creates the virtual machine automatically installs a web server and a web based application.

Full Code

Below is the full end to end codebase


Python

import pulumi
import pulumi_aws as aws

config = pulumi.Config()
project_name = config.get("project_nane", "svk")
project_purpose = config.get("purpose", "route53")
zone_name = config.get("zone_name", "example.com")
use_other_stack = config.get_bool("other_stack", False)

if use_other_stack == True:
  source_stack_name = config.get("sourceStackName")
  source_stack = pulumi.StackReference(source_stack_name)
  vm_ip = source_stack.get_output("public_ip")

mainzone = aws.route53.Zone("mainzone",
    name = zone_name,
)

dev = aws.route53.Zone("dev",
    name="dev." + zone_name,
    tags={
        "Environment": "dev",
    })
dev_ns = aws.route53.Record("dev-ns",
    zone_id=mainzone.zone_id,
    name="dev." + zone_name,
    type=aws.route53.RecordType.NS,
    ttl=30,
    records=dev.name_servers)

if use_other_stack == True:
  www = aws.route53.Record("www",
      zone_id=mainzone.zone_id,
      name="www." + zone_name,
      type=aws.route53.RecordType.A,
      ttl=300,
      #records=["1.2.3.4"]
      records=[vm_ip]
  )
else:
  www = aws.route53.Record("www",
      zone_id=mainzone.zone_id,
      name="www." + zone_name,
      type=aws.route53.RecordType.A,
      ttl=300,
      records=["1.2.3.4"]
  )



devwww = aws.route53.Record("devwww",
    zone_id=dev.zone_id,
    name="www.dev." + zone_name,
    type=aws.route53.RecordType.A,
    ttl=300,
    records=["1.1.1.1"]
)

pulumi.export("Main Nameservers", mainzone.name_servers)
pulumi.export("Dev Nameservers", dev.name_servers)
pulumi.export("stack flag", use_other_stack)

Summary

This is a simple and scalable way of creating DNS zones and making sure that the zones are running in AWS. I have also presented a simple and scalable way to create individual records based on other DNS resources. In addition to this, I have created a conditional way of creating records based on another Pulumi stack and its ouputs.

While DNS is a vast area, this particular example has been kept quite simple so that other more advanced topics can be explored at a later date.

A Simple svelte app

codecowboydotio — Thu, 19 Jun 2025 08:35:13 +0000

Intro

I recently had a need to think about API testing again. I needed to quickly come up with a small web based tester to perform the occasional get request but also a post request with data.

It's a lot simpler for me to just grab a JSON blob of data and send it to an endpoint with a click and evaluate the results.

I decided to use svelte.

I haven't used svelte before but I was surprised by its simplicity and ease of use.
I also admit that I like the fact it has easy templating. :)

TLDR

If you just want the code - it's here:

https://github.com/codecowboydotio/svelte/tree/main/api-tester

High level overview

My app does two things:

It can perform a GET request and display the result
It can perform a POST request and display the result

I find this very useful when I am developing services that are abstracted away in AWS, or some other remote framework and I want to check to see if something is working end to end during the development process.

Install the environment

First I need to create a project directory and then run through some generic project starter questions.

I use npx to create the initial project structure and scaffold code.

npx sv create my-svelte-app
Need to install the following packages:
sv@0.8.11
Ok to proceed? (y) y

┌  Welcome to the Svelte CLI! (v0.8.11)
│
◇  Which template would you like?
│  SvelteKit minimal
│
◇  Add type checking with TypeScript?
│  Yes, using TypeScript syntax
│
◆  Project created
│
◇  What would you like to add to your project? (use arrow keys / space bar)
│  devtools-json
│
◆  Successfully setup add-ons
│
◇  Which package manager do you want to install dependencies with?
│  npm
│
◆  Successfully installed dependencies
│
◇  Project next steps ─────────────────────────────────────────────────────╮
│                                                                          │
│  1: cd my-svelte-app                                                     │
│  2: git init && git add -A && git commit -m "Initial commit" (optional)  │
│  3: npm run dev -- --open                                                │
│                                                                          │
│  To close the dev server, hit Ctrl-C                                     │
│                                                                          │
│  Stuck? Visit us at https://svelte.dev/chat                              │
│                                                                          │
├──────────────────────────────────────────────────────────────────────────╯
│
└  You're all set!

Project Structure

The project structure looks something like this:

I am using a single file layout for my svelte app. This is because I personally find it simpler to do this, and
because it's a small app.

You will note though that there is provision in the project structure to break out components such as CSS and so on.

api-client/
├── src/
│   ├── app.css                 # Global CSS styles
│   ├── app.html                # Main HTML template
│   ├── lib/
│   └── routes/
│       ├── +page.svelte        # Main API client page
├── static/
├── tests/
├── .gitignore
├── package.json
├── README.md
├── svelte.config.js           # SvelteKit configuration
├── tsconfig.json              # TypeScript config (if using TS)
└── vite.config.js             # Vite configuration

The code

As I am using a single file, I will walk through the different components.

Imports

The two imports at the top essentially check if the app is running in a browser or not.

This is described in the svelte docs here:
https://svelte.dev/docs/kit/$app-environment

After the imports, I set some defaults that are used within the app.

<!-- src/routes/+page.svelte -->
<script>
  import { browser } from '$app/environment';
  import { onMount } from 'svelte';

  let url = '';
  let requestData = '{\n  "key": "value"\n}';
  let response = '';
  let loading = false;
  let error = null;
  let showResult = false;
  let method = 'POST'; // Default method selection

Handle form submission

The handle form submission function does most of the heavy lifting within the app.

As the app can handle both GET and POST requests, the first try / catch block checks the method to see if it is a POST. If the method that is received is a POST then the incoming JSON data is parsed for correctness.

  async function handleSubmit() {
    if (!browser) return;

    loading = true;
    error = null;
    showResult = true;

    try {
      // Validate URL
      if (!url) throw new Error('URL is required');

      // For POST requests, validate JSON
      let parsedData;
      if (method === 'POST') {
        try {
          parsedData = JSON.parse(requestData);
        } catch (e) {
          throw new Error(`Invalid JSON: ${e.message}`);
        }
      }

The next statements set the content type, and also include the data portion of the request - only if this is a POST request.

      // Configure fetch options based on the selected method
      const fetchOptions = {
        method: method,
        headers: {
          'Content-Type': 'application/json'
        }
      };

      // Only include body for POST requests
      if (method === 'POST') {
        fetchOptions.body = requestData;
      }

Then make the request:

const fetchResponse = await fetch(url, fetchOptions);

Finally, once the request has returned, be it a GET or a POST, check the headers, and return the response as JSON. If the response is returned not as JSON explcitly, then perform an explicit fetch as json.

      const fetchResponse = await fetch(url, fetchOptions);

      const contentType = fetchResponse.headers.get('content-type');
      if (contentType && contentType.includes('application/json')) {
        const jsonResponse = await fetchResponse.json();
        return JSON.stringify(jsonResponse, null, 2);
      } else {
        const textResponse = await fetchResponse.text();
        return textResponse;
      }
    } catch (err) {
      error = err.message;
      return `Error: ${err.message}`;
    } finally {
      loading = false;
    }
  }

Make the request

The make request function just calls the handle submit function.
The heavy lifting is all done by the other function.

  async function makeRequest() {
    response = await handleSubmit();
  }

Format the json and close the popup box

The next two functions just format the JSON data, and close the object that shows the result by setting it to false.

  function formatJson() {
    try {
      const parsed = JSON.parse(requestData);
      requestData = JSON.stringify(parsed, null, 2);
    } catch (e) {
      // Ignore formatting if JSON is invalid
    }
  }

  function closeResult() {
    showResult = false;
  }
</script>

The HTML

As svelte is a templating language, the script above does the work, but within the HTML and CSS there are sections that both make the request but also handle the result.

Svelte is sort of cool in the way that it handles this.

Form Design

The goal of designing the form was to make it easy to use. The form changes based on the selected HTTP method.
The URL input is always visible, but the JSON body textarea only appears for POST requests:

{#if method === 'POST'}
  <div class="form-group">
    <label for="data">
      Request Body (JSON)
      <button type="button" class="format-btn" on:click={formatJson}>Format</button>
    </label>
    <textarea
      id="data"
      bind:value={requestData}
      placeholder='"key": "value"'
      rows="10"
      required
    ></textarea>
  </div>
{/if}

The "Format" button will automatically prettify JSON input – a small thing that makes the tool easier to use.

The Popup Response Window

Instead of a traditional sidebar or inline display, the response appears in a modal popup. This makes the response easy to read, and also means that you need to take action before hitting the request button again.

The popup structure is straightforward:

{#if showResult}
  <div class="popup-overlay" on:click={closeResult}>
    <div class="popup-window" on:click|stopPropagation>
      <div class="popup-header">
        <h2>API Response</h2>
        <button class="close-btn" on:click={closeResult}>×</button>
      </div>
      <!-- Response content here -->
    </div>
  </div>
{/if}

I used on:click|stopPropagation on the popup window to prevent it from closing when users click inside it, but clicking outside the overlay dismisses the popup – a UX pattern most users expect.

Loading States and Error Handling

There are three distinct states for the response area:

Loading: Shows a spinning animation while the request is in progress
Error: Displays error messages in a red-bordered container
Success: Shows the formatted response in a code block

{#if loading}
  <div class="loading">
    <div class="spinner"></div>
    <p>Sending request...</p>
  </div>
{:else if error}
  <div class="error">
    <p>{error}</p>
  </div>
{:else}
  <pre>{response}</pre>
{/if}

The loading spinner uses a simple CSS animation. This animation just spins. :)

@keyframes spin {
  0% { transform: rotate(0deg); }
  100% { transform: rotate(360deg); }
}

How to Use It

Using my API client is straightforward:

Enter the API endpoint URL in the first field
Select GET or POST from the dropdown
For POST requests, add your JSON payload in the textarea (use the Format button to prettify it)
Click "Send Request" and watch the response appear in the popup

The tool handles the rest – parsing responses, formatting JSON, showing loading states, and displaying errors clearly.

Here I am using httpbin.org but any website will work.

The spinner in action!!!

The response being shown inside the pop up window. This means you can easily copy the response and evaluate it.

Complete Source Code

Here's the full SvelteKit component that powers the API client:

<!-- src/routes/+page.svelte -->
<script>
  import { browser } from '$app/environment';
  import { onMount } from 'svelte';

  let url = '';
  let requestData = '{\n  "key": "value"\n}';
  let response = '';
  let loading = false;
  let error = null;
  let showResult = false;
  let method = 'POST';

  async function handleSubmit() {
    if (!browser) return;

    loading = true;
    error = null;
    showResult = true;

    try {
      if (!url) throw new Error('URL is required');

      let parsedData;
      if (method === 'POST') {
        try {
          parsedData = JSON.parse(requestData);
        } catch (e) {
          throw new Error(`Invalid JSON: ${e.message}`);
        }
      }

      const fetchOptions = {
        method: method,
        headers: {
          'Content-Type': 'application/json'
        }
      };

      if (method === 'POST') {
        fetchOptions.body = requestData;
      }

      const fetchResponse = await fetch(url, fetchOptions);

      const contentType = fetchResponse.headers.get('content-type');
      if (contentType && contentType.includes('application/json')) {
        const jsonResponse = await fetchResponse.json();
        return JSON.stringify(jsonResponse, null, 2);
      } else {
        const textResponse = await fetchResponse.text();
        return textResponse;
      }
    } catch (err) {
      error = err.message;
      return `Error: ${err.message}`;
    } finally {
      loading = false;
    }
  }

  async function makeRequest() {
    response = await handleSubmit();
  }

  function formatJson() {
    try {
      const parsed = JSON.parse(requestData);
      requestData = JSON.stringify(parsed, null, 2);
    } catch (e) {
      // Ignore formatting if JSON is invalid
    }
  }

  function closeResult() {
    showResult = false;
  }
</script>

<svelte:head>
  <title>API Client - SvelteKit</title>
  <meta name="description" content="Simple API client for testing REST endpoints" />
</svelte:head>

<main>
  <div class="container">
    <div class="input-section">
      <h1>API Client</h1>

      <form on:submit|preventDefault={makeRequest}>
        <div class="form-group">
          <label for="url">API Endpoint URL</label>
          <input
            type="text"
            id="url"
            bind:value={url}
            placeholder="https://api.example.com/endpoint"
            required
          />
        </div>

        <div class="form-group">
          <label for="method">Request Method</label>
          <select id="method" bind:value={method}>
            <option value="GET">GET</option>
            <option value="POST">POST</option>
          </select>
        </div>

        {#if method === 'POST'}
          <div class="form-group">
            <label for="data">
              Request Body (JSON)
              <button type="button" class="format-btn" on:click={formatJson}>Format</button>
            </label>
            <textarea
              id="data"
              bind:value={requestData}
              placeholder='"key": "value"'
              rows="10"
              required
            ></textarea>
          </div>
        {/if}

        <button type="submit" class="submit-btn" disabled={loading}>
          {loading ? 'Sending Request...' : `Send ${method} Request`}
        </button>
      </form>
    </div>

    {#if showResult}
      <div class="popup-overlay" on:click={closeResult}>
        <div class="popup-window" on:click|stopPropagation>
          <div class="popup-header">
            <h2>API Response</h2>
            <button class="close-btn" on:click={closeResult}>×</button>
          </div>

          <div class="popup-content">
            {#if loading}
              <div class="loading">
                <div class="spinner"></div>
                <p>Sending request...</p>
              </div>
            {:else if error}
              <div class="error">
                <p>{error}</p>
              </div>
            {:else}
              <pre>{response}</pre>
            {/if}
          </div>
        </div>
      </div>
    {/if}
  </div>
</main>

<style>
  /* Global styles and component styling */
  :global(body) {
    font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, Oxygen,
      Ubuntu, Cantarell, 'Open Sans', 'Helvetica Neue', sans-serif;
    margin: 0;
    padding: 0;
    background-color: #f7f9fc;
    color: #333;
  }

  .container {
    min-height: 100vh;
    max-width: 800px;
    margin: 0 auto;
    padding: 2rem;
  }

  .input-section {
    width: 100%;
  }

  h1 {
    color: #2d3748;
    margin-bottom: 1.5rem;
  }

  .form-group {
    margin-bottom: 1.5rem;
  }

  label {
    display: flex;
    justify-content: space-between;
    align-items: center;
    font-weight: 600;
    margin-bottom: 0.5rem;
    color: #4a5568;
  }

  input, textarea, select {
    width: 100%;
    padding: 0.75rem;
    border: 1px solid #e2e8f0;
    border-radius: 0.25rem;
    font-size: 1rem;
    font-family: inherit;
    box-sizing: border-box;
  }

  textarea {
    font-family: monospace;
    resize: vertical;
  }

  .format-btn {
    background: none;
    border: none;
    color: #4299e1;
    cursor: pointer;
    font-size: 0.875rem;
    padding: 0.25rem 0.5rem;
  }

  .submit-btn {
    background-color: #4299e1;
    color: white;
    border: none;
    border-radius: 0.25rem;
    padding: 0.75rem 1.5rem;
    font-size: 1rem;
    font-weight: 600;
    cursor: pointer;
    transition: background-color 0.2s;
  }

  .popup-overlay {
    position: fixed;
    top: 0;
    left: 0;
    width: 100%;
    height: 100%;
    background-color: rgba(0, 0, 0, 0.5);
    display: flex;
    justify-content: center;
    align-items: center;
    z-index: 1000;
  }

  .popup-window {
    background-color: white;
    border-radius: 0.5rem;
    box-shadow: 0 20px 25px -5px rgba(0, 0, 0, 0.1);
    max-width: 90vw;
    max-height: 80vh;
    width: 700px;
    display: flex;
    flex-direction: column;
  }

  .spinner {
    border: 3px solid #e2e8f0;
    border-top: 3px solid #4299e1;
    border-radius: 50%;
    width: 30px;
    height: 30px;
    animation: spin 1s linear infinite;
  }

  @keyframes spin {
    0% { transform: rotate(0deg); }
    100% { transform: rotate(360deg); }
  }
</style>

Conclusion

Building this taught me a lot about SvelteKit's reactivity system and how to create modern user interface. The browser check from $app/environment was crucial for preventing server-side rendering issues, and Svelte's reactive statements made state management incredibly easy.

I encourage you to give svelte a go - I know I'll be using it for other projects!

[Boost]

codecowboydotio — Wed, 14 May 2025 23:00:19 +0000

codecowboydotio

May 12 '25

Creating an MCP server with Anthropic

#python #mcp #ai #api

Comments

9 min read

Creating an MCP server with Anthropic

codecowboydotio — Mon, 12 May 2025 03:29:58 +0000

Intro

There has been a lot of hype on socials recently about MCP or Model Context Protocol. Essentially it's a protocol that allows LLM AI to be extended. Most of the extensions relate to providing context using your own data.

This is what I am going to walk through here.

I will walk through:

Creating an MCP
What works and what works better
My discoveries so far

TLDR

If you just want the code - it's here:

https://github.com/codecowboydotio/mcp-server-examples

Why

The reason I decided to initially play around with MCP was because I had a high level of scepticism regarding it. I've seen a LOT of posts on the internet that talk about it, how it's transformational and so on. I thought to myself "I'm going to look into this, but I'm going to do it bottom up and by writing a server".

High level overview

The following diagram represents a high level overview of what has been built and how it works.

...and don't worry there are more diagrams that show more detail further on :)

Where to start

I started at the following address:

https://modelcontextprotocol.io/introduction

This describes the MCP protocol, but also describes some of the capabilities.
I then decided to to read through the examples here:

https://modelcontextprotocol.io/quickstart/server

This is a great example, because it covered my initial use case almost exactly. It's an example of a weather application that is a REST based API.

After reading through the example, I decided to set up my development environment and start writing some code.

Install the environment

In order to install the environment, you need to grab a copy of the Anthropic client, and then install all of the pre-requisite software.

https://claude.ai/download

Once installed, I needed to get the rest of the development environment up and running. I chose python as the language of choice to develop in - but other SDKs are available (Python, Typescript, Java, Kotlin and C#).

I am also running on windows, so my configuration examples will use windows nomenclature.

Next it's time to install uv for python, it's an alternative package manager, but you can use pip also if you're more comfortable with it.

powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"

Then we create a virtual environment using uv, and activate it (it's the same as any other venv in the pythonic world).

# Create a new directory for our project
uv init swapi
cd swapi

# Create virtual environment and activate it
uv venv
.venv\Scripts\activate

# Install dependencies
uv add mcp[cli] httpx json requests

# Create our server file
new-item swapi.py

Configure the desktop client

In order for your MCP server to be registered, you'll need to configure the desktop client.

Claude has a file called claude_desktop_config.json.

This is where you will define your MCP server. Essentially it's a way for the local desktop application to run your server. You simply point it at your code.

{
    "mcpServers": {
        "swapi": {
            "command": "C:\\Users\\scott\\.local\\bin\\uv",
            "args": [
                "--directory",
                "C:\\users\\scott\\Desktop\\swapi",
                "run",
                "swapi.py"
            ]
        }
    }
}

From here we are ready to start writing some code.

Star Wars API

I am running a local version of the star wars API as a container.
I also inserted myself into the API as an easy way to validate that my MCP server is asking my API and not deriving data from other sources or its trained dataset.

There is a detailed explanation of the API I am using, as this feeds directly into the question of "specific" versus "generalised" for LLM AI models and MCP.

The API that I use is quite specific and has a number of routes:

You will note that each of the routes has two possible requests:

/route/ - Return ALL records
/route/{id} - Return a specific record

All records

All records looks like this:

curl -s 10.1.1.150:3000/people/ | more

[
  {
    "edited": "2014-12-20T21:17:56.891Z",
    "name": "Luke Skywalker",
    "created": "2014-12-09T13:50:51.644Z",
    "gender": "male",
    "skin_color": "fair",
    "hair_color": "blond",
    "height": "172",
    "eye_color": "blue",
    "mass": "77",
    "homeworld": 1,
    "birth_year": "19BBY",
    "image": "luke_skywalker.jpg",
    "id": 1,
    "vehicles": [
      14,
      30
    ],
    "starships": [
      12,
      22
    ],
    "films": [
      1,
      2,
      3,
      6
    ]
  },
  {
    "edited": "2014-12-20T21:17:50.309Z",
    "name": "C-3PO",
    "created": "2014-12-10T15:10:51.357Z",
    "gender": "n/a",
    "skin_color": "gold",
    "hair_color": "n/a",
    "height": "167",
    "eye_color": "yellow",
    "mass": "75",
    "homeworld": 1,
    "birth_year": "112BBY",
    "image": "c-3po.jpg",
    "id": 2,
    "vehicles": [],
    "starships": [],
    "films": [
      1,
      2,
      3,
      4,
      5,
      6
    ]
  },

It is returned as a JSON array.
Note the id element. The API is keyed on id. So when I get an individual people instance, I will see the following:

Single record

A single record looks like this

curl -s 10.1.1.150:3000/people/1 | more

{
  "edited": "2014-12-20T21:17:56.891Z",
  "name": "Luke Skywalker",
  "created": "2014-12-09T13:50:51.644Z",
  "gender": "male",
  "skin_color": "fair",
  "hair_color": "blond",
  "height": "172",
  "eye_color": "blue",
  "mass": "77",
  "homeworld": 1,
  "birth_year": "19BBY",
  "image": "luke_skywalker.jpg",
  "id": 1,
  "vehicles": [
    14,
    30
  ],
  "starships": [
    12,
    22
  ],
  "films": [
    1,
    2,
    3,
    6
  ]
}

My own record

I inserted myself as record id 84.
The idea here is that this is the ultimate test of not only the MCP server, but is also a test of generalised reasoning.

 curl -s http://10.1.1.150:3000/people/84

{
  "edited": "2014-12-20T21:17:56.891Z",
  "name": "Scott",
  "created": "2014-12-09T13:50:51.644Z",
  "gender": "male",
  "skin_color": "fair",
  "hair_color": "blonde",
  "height": "184",
  "eye_color": "blue",
  "mass": "01",
  "homeworld": 1,
  "birth_year": "19BBY",
  "image": "luke_skywalker.jpg",
  "id": 84,
  "vehicles": [],
  "starships": [],
  "films": []
}

Basic MCP server

Initially I thought like a REST API dev, and built the diagram below:

I wrote an MCP server (based on the example server) that pulls back a single person from the star wars database.

The flow

I used the code from the example server, and charged ahead with typical REST based "query for the smallest thing" logic.

A diagram of the MCP tool flow is below:

Client is asked "get swapi character id 1"
MCP tool calls make_request
make_request queries the external API
external API returns a JSON value for a single person with the ID of 1
The make_request function returns RAW JSON data to the MCP tool
The MCP tool called the format_msg function.
The format_msg function makes a secondary call to the API to resolve the planet name
The SWAPI API returns the planet information for a single planet
The format_msg function parses the JSON responses and formats a message
The formatted response is sent back to the client

In the desktop application it looks like this:

The code

The code is below.
The code is relatively straightforward.

from typing import Any
import httpx
import json
import requests
from mcp.server.fastmcp import FastMCP

# Initialize FastMCP server
mcp = FastMCP("swapi")

# Constants
API_BASE = "http://10.1.1.150:3000"
USER_AGENT = "swapi-app/1.0"


async def make_request(url: str) -> dict[str, Any] | None:
    """Make a request to the API with proper error handling."""
    headers = {
        "User-Agent": USER_AGENT,
        "Accept": "application/json"
    }
    async with httpx.AsyncClient() as client:
        try:
            response = await client.get(url, headers=headers, timeout=30.0)
            response.raise_for_status()
            return response.json()
        except Exception:
            return None


def format_msg(json_string):
    """Format an alert feature into a readable string."""
    props = json_string
    homeworld=json_string['homeworld']
    homeworld_url="http://10.1.1.150:3000/planets/" + str(homeworld)
    homeworld_name = requests.get(homeworld_url)
    homeworld_json=json.loads(homeworld_name.text)

    return f"""
Name: {props.get('name', 'Unknown')}
Gender: {props.get('gender', 'Unknown')}
Hair Colour: {props.get('hair_color', 'Unknown')}
Homeworld: {homeworld_json.get('name', 'Unknown')}
"""


@mcp.tool()
async def get_swapi_character(id: str) -> str:
    """Get name from swapi api.

    Args:
        id: exact name match
    """
    url = f"{API_BASE}/people/{id}"
    data = await make_request(url)

    if not data:
        return "Unable to fetch data from API."

    #alerts = [format_alert(feature) for feature in data["features"]]
    msg = format_msg(data)
    return msg

if __name__ == "__main__":
    # Initialize and run the server
    mcp.run(transport='stdio')

Note that the MCP tool is making a specific request for an ID.

The results

The results are very interesting. There are differences between specific searching and just dumping large datasets into the LLM and getting the LLM to do the heavy lifting via more generalised searching.

Specific searching

When searching for a specific character, I get a single set of results. This is a great start and it matches my single record above. Note that the response is formatted in the same way that my function returns it.

Next I retrieve another character with an ID of 84. This corresponds to the character that I added to the database of the API. It's me!

The interesting part here, is that while my MCP server has been used to get the result, additional context and reasoning has been applied to this answer. The additional reasoning says that this character is also from planet 1 (Tatooine) and is not fetured as part of the official star wars canon of characters.

When I ask a second time, the answer becomes more refined, and specific.

Finally, I ask claude to give me characters that appear in my search but do not appear in other sources. Note that this answer does not use my MCP server.

Generalised searching

I can also perform more generalised searching using the results that I have so far.

I can compare the characters that I have.
This made me wonder what would happen if I had a much larger dataset.

I altered the code to pull in ALL people.

I changed the url on my mcp server to be /people rather than requiring a specific ID.

@mcp.tool()
#async def get_swapi_character(id: str) -> str:
async def get_swapi_character():
    """Get name from swapi api.

    Args:
        id: exact name match
    """
    url = f"{API_BASE}/people/"
    data = await make_request(url)

    if not data:
        return "Unable to fetch data from API."

    #alerts = [format_alert(feature) for feature in data["features"]]
    #msg = format_msg(data)
    #return "\n---\n".join(msg)
    return data

if __name__ == "__main__":
    # Initialize and run the server
    mcp.run(transport='stdio')

This allows me to perform a much less specific search. I automatically pull back ALL star wars characters, and then claude searches the result set to find a specific things that I am searching for.

Note that above the search uses a more generalised search through the entire dataset to find exactly what I want. The search still uses my MCP server even though it is more generalised in nature.

The response section of my search still sends a blank search request because I don't need anything to be sent, but ALL possible star wars characters are returned.

This is an important concept. The larger the data source you send to the model, the more you are able to search through it and combine answers.

Combined answers

Now that I have a larger data set, I can combine answers, and splice the data however I need to.

This answer does not require running the MCP server again, as the original result set is still within the same conversation.

Conclusion

Writing my own MCP server was very instructive and helped me a lot with being able to understand the MCP protocol a lot. I learned that having a larger data set will give you better answers. I also learned that this pattern is almost like an anti pattern of the current API design and thought process. Rather than querying an API for the smallest possible unit of data (like /people/1) it's better from an LLM perspective to import a larger and more complete set of data. This allows thorough searching and splicing of the datasets by the LLM.

Other than this, it was a lot of fun to do, and has helped me to be able to discern more useful AI / LLM / MCP content that I see in other places.

I encourage everyone to have a go and give this a try.

Decentralized programming with libp2p

codecowboydotio — Fri, 01 Sep 2023 21:27:32 +0000

Distributed everything

I thought that I would write a quick post about running libp2p. If you haven't come across it yet, then you should definitely have a look at it.

The main website is here: https://libp2p.io/

Libp2p supports multiple languages, rust, golang, javascript and so on, today I will be using javascript.

Why?

I started to explore libp2p a little while ago. The main reason was personal interest. When I started playing around with it, I figured out how useful it was for a bunch of projects I'm working on (namely project tetsuo - https://tetsuo.dev).

What is it - what does it do for me?

Libp2p is a modular networking stack that you can use in your projects. It allows a developer to quickly and easily create a project that has safe, easy, decentralized networking.

That's right decentralized.

This means that multiple libp2p nodes can discover each other without needing a centralized gateway or broker (you can have this of course, but decentralized interests me more).

If you have two nodes, they will discover each other.

If you have three nodes, they will also discover each other.

Decentralized applications

If you are following along, this means that you can run a completely decentralized topology. This also means that your entire application can be decentralized.

This is important for applications where you may only want clients to connect to each other.

It is also important where you want clients to discover and automatically connect to each other.

Some of the use cases here are for things like:

Completely decentralized peer to peer applications.
Cluster services.
Disconnected applications.

...the possibilities are endless.

What am I showing you today?

Today, I am going to walk through the code to be able to get a basic node to node automatic discovery happening.

Set up a project

To start with, I need to initialize a project. Since I'm using javascript, we will go ahead and start.

# create a directory for the project
mkdir test
cd test

# Initialize the project
npm init es6

Accept the defaults for your npm init. This will create a package.json file. We are good to go from here.

We are initializing the project as a module.

Get the scaffold running

To begin with, we will be creating a scaffold. All this will do at this point is create a libp2p node.

Install the pre-requisites

npm install @libp2p/tcp 
npm install libp2p

Place the code below into a file called test.js

import { tcp } from '@libp2p/tcp'
import { createLibp2p } from 'libp2p'

const createNode = async () => {
  const node = await createLibp2p({
    addresses: { listen: ['/ip4/0.0.0.0/tcp/0'] },
    transports: [ tcp() ],
  }) //end node

  return node
} //end createNode function

;(async () => {
  const [node] = await Promise.all([
    createNode(),
  ]) //end node creation as promise

  console.log('listening on addresses:')
    node.getMultiaddrs().forEach((addr) => {
      console.log(addr.toString())
  }) //end logging
})()

The code creates a new libp2p node that listens on all addresses, using TCP.

const createNode = async () => {
  const node = await createLibp2p({
    addresses: { listen: ['/ip4/0.0.0.0/tcp/0'] },
    transports: [ tcp() ],
  }) //end node

  return node
} //end createNode function

Then there is a main function that creates the node and prints out all of the addresses that the node is listening on.

;(async () => {
  const [node] = await Promise.all([
    createNode(),
  ]) //end node creation as promise

  console.log('listening on addresses:')
    node.getMultiaddrs().forEach((addr) => {
      console.log(addr.toString())
  }) //end logging
})()

You can run this using the command

node test.js
# node test.js
listening on addresses:
/ip4/127.0.0.1/tcp/33389/p2p/12D3KooWCR4TBnsBq6S7gkJuxSMFCBsQmXT4V4Z5hpEwbixsE6Va
/ip4/10.1.1.150/tcp/33389/p2p/12D3KooWCR4TBnsBq6S7gkJuxSMFCBsQmXT4V4Z5hpEwbixsE6Va
/ip4/192.168.224.129/tcp/33389/p2p/12D3KooWCR4TBnsBq6S7gkJuxSMFCBsQmXT4V4Z5hpEwbixsE6Va

Node discovery

So far we have a single libp2p node that listens. This is not that useful. Next step is to allow our node to discover other nodes on the network.

Today, we will use the multicast DNS module for this. The multicast DNS module is limited to local networks, but works well for testing and understanding.

Install the multicast DNS module.

npm install @libp2p/mdns

Update the test.js file with the following content.

import { mdns } from '@libp2p/mdns'
import { tcp } from '@libp2p/tcp'
import { createLibp2p } from 'libp2p'

const createNode = async () => {
  const node = await createLibp2p({
    addresses: { listen: ['/ip4/0.0.0.0/tcp/0'] },
    transports: [ tcp() ],
    peerDiscovery: [ mdns() ] //default discovery period 10 seconds
  }) //end node

  return node
} //end createNode function

;(async () => {
  const [node] = await Promise.all([
    createNode(),
  ]) //end node creation as promise

  console.log('listening on addresses:')
    node.getMultiaddrs().forEach((addr) => {
      console.log(addr.toString())
  }) //end logging

  node.addEventListener('peer:discovery', (evt) => console.log('Discovered:', evt.de
tail.id.toString()))
})()

We have added some pieces to our codebase. We have added a peerDiscovery piece into the createNode async function. This essentially enables peer discovery using multicast DNS.

There are other options for peer discovery, but in this example, I am using multicast DNS.

const createNode = async () => {
  const node = await createLibp2p({
    addresses: { listen: ['/ip4/0.0.0.0/tcp/0'] },
    transports: [ tcp() ],
    peerDiscovery: [ mdns() ] //default discovery period 10 seconds
  }) //end node

This means that your node will now automatically listen on all addresses, and will perform a peer discovery search via multicast every 10 seconds. The 10 second time is fine for testing, but can be extended for real scenarios.

At the end of our main async function we place the following.

;(async () => {
  node.addEventListener('peer:discovery', (evt) => console.log('Discovered:', evt.detail.id.toString()))
})()

This piece of code adds a listener that listens specifically for peer discovery events.

Running the new code

Let's run the new codebase.

# node test.js
listening on addresses:
/ip4/127.0.0.1/tcp/33667/p2p/12D3KooWC5c39mad27di37ubxUymmoohVhWNedMbPrUzYN7tiV2R
/ip4/10.1.1.150/tcp/33667/p2p/12D3KooWC5c39mad27di37ubxUymmoohVhWNedMbPrUzYN7tiV2R
/ip4/192.168.224.129/tcp/33667/p2p/12D3KooWC5c39mad27di37ubxUymmoohVhWNedMbPrUzYN7tiV2R

At this point, it looks exactly the same as the last time we ran it. The reason for this is that we need to run two nodes in order for us to discover anything.

Let's fire up a second instance of test.js in a separate window

node1 output

# node test.js
listening on addresses:
/ip4/127.0.0.1/tcp/33667/p2p/12D3KooWC5c39mad27di37ubxUymmoohVhWNedMbPrUzYN7tiV2R
/ip4/10.1.1.150/tcp/33667/p2p/12D3KooWC5c39mad27di37ubxUymmoohVhWNedMbPrUzYN7tiV2R
/ip4/192.168.224.129/tcp/33667/p2p/12D3KooWC5c39mad27di37ubxUymmoohVhWNedMbPrUzYN7tiV2R

node2 output

# node test.js
listening on addresses:
/ip4/127.0.0.1/tcp/35725/p2p/12D3KooWPJSvTFo7jhaLACx1vZVKq7bHhNHKXALXWCjKqFu5LK47
/ip4/10.1.1.150/tcp/35725/p2p/12D3KooWPJSvTFo7jhaLACx1vZVKq7bHhNHKXALXWCjKqFu5LK47
/ip4/192.168.224.129/tcp/35725/p2p/12D3KooWPJSvTFo7jhaLACx1vZVKq7bHhNHKXALXWCjKqFu5LK47
Discovered: 12D3KooWC5c39mad27di37ubxUymmoohVhWNedMbPrUzYN7tiV2R

You will note that the second node has automatically discovered the first node.

Discovered: 12D3KooWC5c39mad27di37ubxUymmoohVhWNedMbPrUzYN7tiV2R

node1 output - check again

If you check the output of node1 again, you will note that it too has automatically discovered the other node.

# node test.js
listening on addresses:
/ip4/127.0.0.1/tcp/33667/p2p/12D3KooWC5c39mad27di37ubxUymmoohVhWNedMbPrUzYN7tiV2R
/ip4/10.1.1.150/tcp/33667/p2p/12D3KooWC5c39mad27di37ubxUymmoohVhWNedMbPrUzYN7tiV2R
/ip4/192.168.224.129/tcp/33667/p2p/12D3KooWC5c39mad27di37ubxUymmoohVhWNedMbPrUzYN7tiV2R
Discovered: 12D3KooWPJSvTFo7jhaLACx1vZVKq7bHhNHKXALXWCjKqFu5LK47

Each node is broadcasting and discovering it's peer on the network.

Full code

The full code is listed below.

import { mdns } from '@libp2p/mdns'
import { tcp } from '@libp2p/tcp'
import { createLibp2p } from 'libp2p'

const createNode = async () => {
  const node = await createLibp2p({
    addresses: { listen: ['/ip4/0.0.0.0/tcp/0'] },
    transports: [ tcp() ],
    peerDiscovery: [ mdns() ] //default discovery period 10 seconds

//    streamMuxers: [ yamux(), mplex() ],
//    connectionEncryption: [ noise() ],
  }) //end node

  return node
} //end createNode function

;(async () => {
  const [node] = await Promise.all([
    createNode(),
  ]) //end node creation as promise

  console.log('listening on addresses:')
    node.getMultiaddrs().forEach((addr) => {
      console.log(addr.toString())
  }) //end logging

  node.addEventListener('peer:discovery', (evt) => console.log('Discovered:', evt.detail.id.toString()))

// ADD YOUR APPLICATION CODE HERE

})()

What Now?

This project is a scaffold of how you can set up node discovery using libp2p. Now that you have a scaffold running, you can add your application code to the scaffold.

In my case, I am writing a cluster mechanism for application servers to automatically exchange configuration events. Every member of my application cluster will automatically receive updates to its configuration no matter which node receives the update.

In order to do this, I will be using a peer to peer message bus (also another feature of libp2p). Look out for my next post where I walk through how to get pub sub events working!

Pulumi - creating cloud infrastructure using yaml

codecowboydotio — Tue, 02 Aug 2022 12:29:00 +0000

Intro

I've been fooling around with pulumi for a bit now, and thought I would write about it here.

What is it?

Pulumi is an infrastructure as code framework with a twist. You can use any language you like in order to write your code (within reason).

The pulumi tagline is Your cloud. Your language. Your way.

I'm here to say it's true!

Why is it cool?

Pulumi is super cool because it means as a developer I can write infrastructure as code for my cloud based infrastructure in a language I am comfortable with.

This is a HUGE DIFFERENCE to learning an opinionated hybrid language for IAC.

Interestingly it also means I can write tests for my IAC in the same language I'm used to writing in. This is also a massive benefit.

I do not cover it here, but look out for another post soon.

Installation

Installation is simple, I am using AWS and will follow the docs at https://www.pulumi.com/docs/get-started/aws/begin/.

Install pulumi

In order to install pulumi, just as the docs say, use the following command:

curl -fsSL https://get.pulumi.com | sh

You will also need to install your language runtime - good news - for YAML, there is no language to install.

Next configure your AWS account in the normal way that you would

export AWS_ACCESS_KEY_ID=<YOUR_ACCESS_KEY_ID>
export AWS_SECRET_ACCESS_KEY=<YOUR_SECRET_ACCESS_KEY>

Create a new project

In order to get up and going once pulumi is installed you will need to install a new project.

To do this you use the commands:

mkdir quickstart
cd quickstart
pulumi new aws-yaml

By default this will create a project that has two files:

Pulumi.yaml: This contains the program that manages your resources

name: quickstart
runtime: yaml
description: A minimal AWS Pulumi YAML program

resources:
  # Create an AWS resource (S3 Bucket)
  my-bucket:
    type: aws:s3:Bucket

outputs:
  # Export the name of the bucket
  bucketName: ${my-bucket.id}

Pulumi.dev.yaml: This contains project level configuration

config:
  aws:region: ap-southeast-2

Default Project

The default project is fairly simple to understand and creates a bucket with an output.
Let's look at what else we can do that's useful!

A more complex example

A more complex example requires a more complex explanation.
What if I want to create something more akin to a real world use case in AWS?

What if I want to create a VPC, subnet, internet gateway, instance and security groups as a separate stack?

The image below is exactly what we are going to create.

AWS VPC

To create an AWS VPC we have the following code:

name: aws-yaml
runtime: yaml
description: A minimal AWS Pulumi YAML program

configuration:
  project_name:
    type: String
    default: pulumi-yaml
  vpc_name:
    type: String
    default: ${project_name}-vpc
resources:
  main_vpc:
    type: aws:ec2:Vpc
    properties:
      cidrBlock: 10.100.0.0/16
      tags:
        Name: ${vpc_name}

The code above does a few things that are interesting.

It sets up some variables that are used throughout the rest of the codebase.
The variable project_name is set as a type string, and used defaults to pulumi-yaml. This is a standard yaml way of setting a variable.

The variable vpc_name is set as a type string, but uses the previously defined variable, project_name.
This is a good example of how to use variables in yaml if you aren't familiar.

The resources section builds the AWS resources for me. Each resource has a type and properties. These are defined as part of the API documention in pulumi.

In my case, I am creating a VPC, with a CIDR block and tagging it.

To run this code we use the command:

pulumi up -y

This produces:

     Type                 Name          Status
 +   pulumi:pulumi:Stack  aws-yaml-dev  created
 +   └─ aws:ec2:Vpc       main_vpc      created

AWS Subnet

The next step is to create an AWS subnet. This allows me to further carve up the VPC supernet.

The code for the next portion is as follows:

  main_subnet:
    type: aws:ec2:Subnet
    properties:
      vpcId: ${main_vpc.id}
      cidrBlock: 10.100.1.0/24
      mapPublicIpOnLaunch: True
      tags:
        Name: ${project_name}-subnet

Note that the type is different - the object is again a different API type.
The vpcId is important as well. I am using the vpc object name from my previous piece of code, and the calulated id. This allows pulumi to calculate the vpc ID that was created in the previous step.

The output from running this looks like this:

     Type                 Name          Status
     pulumi:pulumi:Stack  aws-yaml-dev
 +   └─ aws:ec2:Subnet    main_subnet   created

Internet Gateway

The next step is to create an internet gateway for my new VPC. This will allow internet connectivity in and out of my VPC.

  internet_gateway:
    type: aws:ec2:InternetGateway
    properties:
      vpcId: ${main_vpc.id}
      tags:
        Name: ${project_name}-igw

Again I use the vpc ID and tag the asset with a tag that includes my project name that I defined above.

When I run this the output is as follows:

     Type                        Name              Status
     pulumi:pulumi:Stack         aws-yaml-dev
 +   └─ aws:ec2:InternetGateway  internet_gateway  created

Route table and Association

The next steps are to create both a route table and associate that route table with the internet gateway created above, as well as a subnet.

  route_table:
    type: aws:ec2:RouteTable
    properties:
      vpcId: ${main_vpc.id}
      routes:
        - cidrBlock: 0.0.0.0/0
          gatewayId: ${internet_gateway.id}
  route_table_assoc:
    type: aws:ec2:RouteTableAssociation
    properties:
      subnetId: ${main_subnet.id}
      routeTableId: ${route_table.id}

Again, the object identifier of vpc ID in object notation is used. I also use the internet gateway ID as well. For the route table association, I use the subnet identifier created above and the route table id.

When I run this, the output is as follows:

     Type                              Name               Status
     pulumi:pulumi:Stack               aws-yaml-dev
 +   ├─ aws:ec2:RouteTable             route_table        created
 +   └─ aws:ec2:RouteTableAssociation  route_table_assoc  created

Security Group

The next step is to create a security group and associate that security group with a VPC.

  security_group:
    type: aws:ec2:SecurityGroup
    properties:
      vpcId: ${main_vpc.id}
      ingress:
        - protocol: -1
          fromPort:  0
          toPort: 0
          cidrBlocks:
            - 0.0.0.0/0
      egress:
        - protocol: -1
          fromPort:  0
          toPort: 0
          cidrBlocks:
            - 0.0.0.0/0
      tags:
        Name: ${project_name}-sg

The output from this step when I run it is:

     Type                      Name            Status
     pulumi:pulumi:Stack       aws-yaml-dev
 +   └─ aws:ec2:SecurityGroup  security_group  created

Instance

The final step here is to create an instance within the newly created VPC and security groups.

  web:
    type: aws:ec2:Instance
    properties:
      ami: ${ubuntu.id}
      instanceType: t3.micro
      tags:
        Name: ${project_name}-instance
variables:
  ubuntu:
    Fn::Invoke:
      Function: aws:ec2:getAmi
      Arguments:
        mostRecent: true
        filters:
          - name: name
            values:
              - ubuntu/images/hvm-ssd/ubuntu-focal-20.04-*
          - name: virtualization-type
            values:
              - hvm
          - name: architecture
            values:
              - x86_64

The first part of the YAML creates an instance, and has the instance identifier and size as well as providing tags for the instance.

The instance identifier is a variable.
The variable is updated with the output of a function.

This occurs dynamically.

The function calls the AWS API and gets the AMI based on the filters provided. This is set as a variable which can then be queried and walked through. Note that I use the identified ubuntu.id. This is the returned value from the function that is placed in the variable.

When I run this, I get the following output:

     Type                 Name          Status
     pulumi:pulumi:Stack  aws-yaml-dev
 +   └─ aws:ec2:Instance  web           created

End to end

End to end the entire yaml file looks like this:

name: aws-yaml
runtime: yaml
description: A minimal AWS Pulumi YAML program




configuration:
  project_name:
    type: String
    default: pulumi-yaml
  vpc_name:
    type: String
    default: ${project_name}-vpc
resources:
  main_vpc:
    type: aws:ec2:Vpc
    properties:
      cidrBlock: 10.100.0.0/16
      tags:
        Name: ${vpc_name}
  main_subnet:
    type: aws:ec2:Subnet
    properties:
      vpcId: ${main_vpc.id}
      cidrBlock: 10.100.1.0/24
      mapPublicIpOnLaunch: True
      tags:
        Name: ${project_name}-subnet
  internet_gateway:
    type: aws:ec2:InternetGateway
    properties:
      vpcId: ${main_vpc.id}
      tags:
        Name: ${project_name}-igw
  route_table:
    type: aws:ec2:RouteTable
    properties:
      vpcId: ${main_vpc.id}
      routes:
        - cidrBlock: 0.0.0.0/0
          gatewayId: ${internet_gateway.id}
  route_table_assoc:
    type: aws:ec2:RouteTableAssociation
    properties:
      subnetId: ${main_subnet.id}
      routeTableId: ${route_table.id}
  security_group:
    type: aws:ec2:SecurityGroup
    properties:
      vpcId: ${main_vpc.id}
      ingress:
        - protocol: -1
          fromPort:  0
          toPort: 0
          cidrBlocks:
            - 0.0.0.0/0
      egress:
        - protocol: -1
          fromPort:  0
          toPort: 0
          cidrBlocks:
            - 0.0.0.0/0
      tags:
        Name: ${project_name}-sg
  web:
    type: aws:ec2:Instance
    properties:
      ami: ${ubuntu.id}
      instanceType: t3.micro
      tags:
        Name: ${project_name}-instance
variables:
  ubuntu:
    Fn::Invoke:
      Function: aws:ec2:getAmi
      Arguments:
        mostRecent: true
        filters:
          - name: name
            values:
              - ubuntu/images/hvm-ssd/ubuntu-focal-20.04-*
          - name: virtualization-type
            values:
              - hvm
          - name: architecture
            values:
              - x86_64

When I run it from the beginning, I get the following output:

     Type                              Name               Status
 +   pulumi:pulumi:Stack               aws-yaml-dev       created
 +   ├─ aws:ec2:Vpc                    main_vpc           created
 +   ├─ aws:ec2:Instance               web                created
 +   ├─ aws:ec2:SecurityGroup          security_group     created
 +   ├─ aws:ec2:Subnet                 main_subnet        created
 +   ├─ aws:ec2:InternetGateway        internet_gateway   created
 +   ├─ aws:ec2:RouteTable             route_table        created
 +   └─ aws:ec2:RouteTableAssociation  route_table_assoc  created

Resources:
    + 8 created

Duration: 39s

Cleanup

Cleanup is just as easy.

pulumi delete -y

This deletes all of the resources that have been created.

     Type                              Name               Status
 -   pulumi:pulumi:Stack               aws-yaml-dev       deleted
 -   ├─ aws:ec2:RouteTableAssociation  route_table_assoc  deleted
 -   ├─ aws:ec2:RouteTable             route_table        deleted
 -   ├─ aws:ec2:Subnet                 main_subnet        deleted
 -   ├─ aws:ec2:SecurityGroup          security_group     deleted
 -   ├─ aws:ec2:InternetGateway        internet_gateway   deleted
 -   ├─ aws:ec2:Instance               web                deleted
 -   └─ aws:ec2:Vpc                    main_vpc           deleted

Resources:
    - 8 deleted

Duration: 1m2s

Conclusion

Pulumi rocks!

I don't know how else to say it.

I can use a multitude of languages to create my cloud resources - languages that I'm familiar with and like.

This example is YAML - which isn't quite a language, but has some characteristics of a language. I learned a few things about YAML and syntax through this process, and hopefully you did too.

Look out for my next post where I will walk through pulumi and python.