Programming Jan/Feb 2021

I was pretty busy programming at the start of 2021 across a few different languages. Let’s jump right in!

C#

I’m nearing the end of the GameDev.tv online RTS course, and it’s been a lot of fun. Since last time we added player colors to the units, a minimap that can be used to move around the screen, new units, and a Lobby UI. I’m a few lessons away from being able to create binaries I can use to play online with others or via Steam.

MS MakeCode

I wanted to try a new electronic project with Stella, so I fired up the tutorial for the Circuit Playground chairs swing ride. I had her cut out the figures that would go on the ride. Then I used the glue gun to put everything together. After all the parts were ready, I had Stella do the programming since the kids are used to programming in Scratch and MS MakeCode is basically Scratch plus the ability to do Javascript.

Python

Python Morsels

I completed my final assignment for the Python Morsels exercises. This time we had to take in a list of timestamps and sum them together. This time it wasn’t only a great exercise to learn and practice with Python, but it’s also a utility I can use in various projects. I immediately realized it could have made summing up the times for my end of year video games blog post a lot easier. Here’s the solution I came up with:

import math 


def figure_out_base_sixty(number: int) -> (int, int): 
    """Figure out the next number up if I have more than 59 seconds or minutes.""" 
    if number > 59: 
        return math.floor(number/60), number % 60 
    else: 
        return 0, number 


def sum_timestamps(timestamps: list) -> str: 
    """Accept a list of timestamps (in the format of MM:SS or HH:MM:SS) and return a timestamp that is the sum of all 
    given times. 

    :param timestamps: A list of timestamps. 
    :type timestamps: list 
    :returns: Timestamp sum of all times in the list. 
    """ 
    split_time = [timestamp.split(':') for timestamp in timestamps] 
    hours = [] 
    minutes = [] 
    seconds = [] 
    for timestamp in split_time: 
        if len(timestamp) == 2: 
            minutes.append(int(timestamp[0])) 
            seconds.append(int(timestamp[1])) 
        else: 
            hours.append((int(timestamp[0]))) 
            minutes.append(int(timestamp[1])) 
            seconds.append(int(timestamp[2])) 
    total_hours_step_1 = sum(hours) 
    total_minutes_step_1 = sum(minutes) 
    total_seconds_step_1 = sum(seconds) 
    minutes_to_add, total_seconds_final = figure_out_base_sixty(total_seconds_step_1) 
    total_minutes_step2 = total_minutes_step_1 + minutes_to_add 
    if total_minutes_step2 > 59 or hours: 
        calculated_hours, total_minutes = figure_out_base_sixty(total_minutes_step2) 
        final_hours = calculated_hours + total_hours_step_1 
        return f"{final_hours}:{total_minutes:02d}:{total_seconds_final:02d}" 
    else: 
        total_minutes = total_minutes_step2 
        return f"{total_minutes}:{total_seconds_final:02d}"

After all this time of learning, my solution was actually pretty close to Trey’s solution, and that really made me feel great.

Pybites

In the same Humble Bundle where I got the Python Morsels exercise credits, I also got access to Pybites. I’ve done about 7 or 8 of them so far. The platform is nice, with a built-in Python interpreter and a good community. But if I compare it to Python Morsels, I don’t like it as much for learning. They give you a problem and provide unit tests to make sure your code works. But, unlike Trey – if you need help the solution is simply stated. Trey’s solutions, on the other hand, contain a lot of commentary that help to explain why it’s the best answer. He also covers answers that work, but aren’t as pythonic. His bonus questions build on the base problem in a way that really encourages learning. Pybites isn’t bad, but after doing Python Morsels – if I were going to pay for one of them, I’d pay for Python Morsels.

Civilization VI Play by Cloud Webhook

I’m a little late to this, but in the final weekend of February 2021 my brothers and I started playing Civilization VI using Play by Cloud. Previously, the only way to play asynchronously was to use an external service like Play Your Damn Turn. (We used to use another one with Civ V, but Play your Damn Turn works with Civ V, Civ VI, and Beyond Earth). Play Your Damn Turn (PYDT) has a neat little app that runs on your computer to let you know when there’s another turn. Play By Cloud, which Firaxis integrated into Civilization VI in a February 2019 update, just uses Steam updates. Dan and Dave have been using Play by Cloud for their newest games, but this weekend they invited me to start a game on there, too. For some reason, Steam updates didn’t quite work all that well to let me know when it was time to take my turn, but I found out you can set up webhooks to get informed by Civilization when there’s a new turn available. Almost all the examples I found online involved setting up the webhooks to push alerts to Discord. I’ve got a Matrix server, so I spent the weekend writing up a program to handle it for us.

When I was working on it, I didn’t create a perfectly sanitized program without any info I wouldn’t want out there, so I’m not ready to put it on Github, but I can share some sanitized code here.

When I was trying to develop, it took some searching to find out what data is sent by Civilization VI. It sends JSON:

{
"value1": "the name of your game",
"value2": "the player's Steam name",
"value3": "the turn number"
}

While working on the code, I found out that you can also set up webhooks with PYDT. They actually send even more data (with actually useful names!):

{
"gameName": "the name of your game",
"userName": "the user's Steam username",
"round": the round number (int),
"civName": "the name of your civilization",
"leaderName": "the name of your civ leader (and for some of them the attribute)"
}

The first thing you need when dealing with webhooks, is a place that can handle a POST command on the net. This was absolutely perfect with Flask as Django would be way heavier than we need to just accept some POST requests.

from flask import Flask, request, Response, jsonify 
import json 
import logging 

import matrix_bot 

app = Flask( __name__ ) 
flask_matrix_bot = matrix_bot.MatrixBot() 
most_recent_games = dict() 
logging.basicConfig(level=logging.DEBUG, format='%(levelname)s- %(asctime)s - %(message)s') 

try: 
    with open('most_recent_games.json', 'r') as file: 
        most_recent_games = json.load(file) 
        logging.debug("JSON file loaded.") 
except FileNotFoundError: 
    logging.warning("Prior JSON file not found. If this is your first run, this is OK.") 


def player_name_to_matrix_name(player_name: str) -> str: 
    if player_name == "One Oh Eight": 
        return "Dan" 
    elif player_name == "TheDJOtaku": 
        return "Eric" 
    elif player_name == "Wedge": 
        return "David" 
    else: 
        return player_name 


@app.route('/webhook', methods=['POST']) 
def respond(): 
    logging.debug(f'JSON from Play By Cloud: {request.json}') 
    game_name = request.json.get('value1') 
    player_name = player_name_to_matrix_name(request.json.get('value2')) 
    turn_number = request.json.get('value3') 
    if game_name in most_recent_games.keys(): 
        if most_recent_games[game_name]['player_name'] != player_name: 
            logging.debug("Game exists, but this is not a duplicate") 
            message = f"Hey, {player_name}, it's your turn in {game_name}. The game is on turn {turn_number}" 
            flask_matrix_bot.main(message) 
            most_recent_games[game_name] = {'player_name': player_name, 'turn_number': turn_number} 
        else: 
            logging.debug("Game exists and this is a duplicate entry.") 
    else: 
        most_recent_games[game_name] = {'player_name': player_name, 'turn_number': turn_number} 
        logging.debug("New game.") 
        message = f"Hey, {player_name}, it's your turn in {game_name}. The game is on turn {turn_number}" 
        flask_matrix_bot.main(message) 
    with open('most_recent_games.json', 'w') as file: 
        json.dump(most_recent_games, file) 
    return Response(status=200) 


@app.route('/pydt', methods=['POST']) 
def respond_pydt(): 
    logging.debug(f'JSON from PYDT: {request.json}') 
    game_name = request.json.get('gameName') 
    player_name = player_name_to_matrix_name(request.json.get('userName')) 
    turn_number = request.json.get('round') 
    civ_name = request.json.get('civName') 
    leader_name = request.json.get('leaderName') 
    message = f"Hey, {player_name}, {leader_name} is waiting for you to command {civ_name} in {game_name}. " \ 
              f"The game is on turn {turn_number}" 
    flask_matrix_bot.main(message) 
    most_recent_games[game_name] = {'player_name': player_name, 'turn_number': turn_number} 
    with open('most_recent_games.json', 'w') as file: 
        json.dump(most_recent_games, file) 
    return Response(status=200) 


@app.route('/recent_games', methods=['GET']) 
def return_recent_games(): 
    return jsonify(most_recent_games)

First, lines 12-17, I check for the most recent games that I’ve saved out to a JSON file. This allows for persistence if I have to stop the program or it crashes or whatever. Then, lines 20-28, I also created a function to convert from Steam screennames to Matrix names so that when the Matrix bot pushes the data, it’ll come out as a mention to the user and draw their attention to the turn they need to play. Here’s what it looks like:

Hey, Eric, Montezuma is waiting for you to command Aztec in Gathering Storm! 1. The game is on turn 59

Hey, Eric, Gilgamesh is waiting for you to command Sumeria in Mesas Play . The game is on turn 161

Hey, Eric, it's your turn in Eric and Dan Duel. The game is on turn 9

You can see in the first two I can make use of the extra data PYDT gives us. The last one is Play By Cloud, giving us less to work with.

I create two endpoints – one for Civilization VI (starting at line 31) and one for PYDT (line 55). The Civilization VI endpoint (webhook) has deduplication code because if each person has it set to push out the JSON on all turns, it would end up repeating the alert.

I also added the recent_games endpoints so that I could create another matrix bot that would listen for commands and let users check up on their games.

"""Provide ability to listen for commands from Matrix.""" 

import asyncio 
import json 
import logging 
from nio import AsyncClient 
import requests 

# testing 
# URL = "http://localhost:5000/recent_games" 
# production 
URL = "<YOUR SERVER URL>" 

logging.basicConfig(level=logging.INFO, format='%(levelname)s - %(asctime)s - %(message)s') 


class ListenerMatrixBot: 
    """A bot to send alerts about the game to Matrix""" 
    def __init__ (self): 
        try: 
            with open('matrix.conf') as file: 
                self.config = json.load(file) 
                logging.debug("Listener Matrix Config loaded.") 
                file.close() 
        except FileNotFoundError: 
            logging.warning(f"Settings not found.") 

    async def login(self): 
        client = AsyncClient(self.config.get('server'), self.config.get('username')) 
        response = await client.login(password=self.config.get("password")) 
        logging.info(f"Listener Login response: {response}") 
        logging.debug(f"Listener Room would be: {self.config.get('room')}") 
        return client 

    @staticmethod 
    def get_current_games(): 
        """Get the list of games from the recent games endpoint. 

        :returns: A dictionary of the games, next player, and turn number. 
        """ 
        response = requests.get(URL) 
        return dict(response.json()) 

    def format_current_games(self): 
        """Format the list of current games for display in Matrix server.""" 
        return_text = "Here is a list of the games currently known about on the server:\n" 
        response_dictionary = self.get_current_games() 
        for key in response_dictionary: 
            game = key 
            player = response_dictionary[key].get('player_name') 
            turn_number = response_dictionary[key].get('turn_number') 
            return_text += f"{game} awaiting turn {turn_number} by {player}\n" 
            logging.debug(return_text) 
        return return_text 

    def format_blame_games(self, player_name: str) -> str: 
        number_of_games = 0 
        return_text = "" 
        response_dictionary = self.get_current_games() 
        for key in response_dictionary: 
            game = key 
            turn_number = response_dictionary[key].get('turn_number') 
            if response_dictionary[key].get('player_name') == player_name: 
                return_text += f"{game} awaiting turn {turn_number} by {player_name}\n" 
                number_of_games += 1 
        if number_of_games > 0: 
            return f"There are {number_of_games} games waiting for {player_name} to take their turn:\n" + return_text 
        else: 
            return f"There aren't any games waiting for {player_name}. Great job!" 

    async def main(self): 
        my_client = await self.login() 
        with open('next_batch', 'r') as next_batch_token: 
            my_client.next_batch = next_batch_token.read() 
        while True: 
            sync_response = await my_client.sync(30000) 
            with open('next_batch', 'w') as next_batch_token: 
                next_batch_token.write(sync_response.next_batch) 
            if len(sync_response.rooms.join) > 0: 
                joins = sync_response.rooms.join 
                for room_id in joins: 
                    for event in joins[room_id].timeline.events: 
                        if hasattr(event, 'body'): 
                            if event.body.startswith("!Civ_Bot current games"): 
                                data_to_send = self.format_current_games() 
                                logging.debug(data_to_send) 
                                content = {"body": data_to_send, "msgtype": "m.text"} 
                                await my_client.room_send(room_id, 'm.room.message', content) 
                            elif event.body.startswith("!Civ_Bot help"): 
                                data_to_send = ("""Current Commands: 
                            !Civ_Bot help - this message 
                            !Civ_Bot current games - the list of games Civ_Bot currently knows about. 
                            !Civ_Bot blame <Matrix Username> - list of games waiting on that person.""") 
                                content = {"body": data_to_send, "msgtype": "m.text"} 
                                await my_client.room_send(room_id, 'm.room.message', content) 
                            elif event.body.startswith("!Civ_Bot blame"): 
                                player = event.body.lstrip("!Civ_Bot blame ") 
                                data_to_send = self.format_blame_games(player) 
                                logging.debug(data_to_send) 
                                content = {"body": data_to_send, "msgtype": "m.text"} 
                                await my_client.room_send(room_id, 'm.room.message', content) 


my_matrix_bot = ListenerMatrixBot() 
asyncio.run(my_matrix_bot.main())

For the weekend or evening, I could get to the game soon after getting notified. But on weekdays, my brothers might play their turns while I’m at work and then by the time I get home (or even a few days later) I might forget whether there are games waiting for me. So I set up the listener to allow me to either ask it about all the games outstanding or just my own by using !Civ_Bot blame Eric.

Overall, I was pretty darned happy with how it came out for just a weekend’s worth of work.

Impractical Python Chapter Projects

I did a few more chapters of the Impractical Python book. Specifically we worked on Genetic Algorithms for breeding mice and cracking safes; Haiku in which we implemented syllable counting and Natural Language Processing with NLTK. You can see the code I wrote and what I learned in these chapters by going to my Github repo.

ELDonation Tracker

I refactored all the Python files into submodules to better separate what each file does. It is much cleaner now. Plus I finally released 6.0 – there will be a separate blog post covering that.

NASA Background Photo Downloader

Sometimes there’s more than one photo per day on the NASA feed so I changed the code to grab the last 3 images.

Scratch Jr

I also finished the Scratch Jr book with Stella. I enjoyed the book. It appears Goodreads is having some issues now, but I’ll try to remember to add a link to my review when I’m able to.