DEV Community: Adam.S

Custom Corporate Holidays with holidays package

Adam.S — Thu, 06 Nov 2025 02:15:45 +0000

I am currently working on a newish personal project. My plan is to create a Python version of a Google Script app
that I am running. I wanted to find a way to determine if a given date is a national holiday. This requires taking into account that National Holidays that fall on a Sunday are moved to the following Monday or to
Tuesday if Monday is already a National Holiday. I also needed to support adding Company holidays. This is important as
it impacts some calculations. Enter the Python package: holidays

The Challenge

I needed to find a way that would allow me to check if a given date was a national holiday in Japan, and also provide support for additional corporate holidays.

In Japan, Christmas is not a national holiday, but the company I work for sets Christmas as a company holiday. Additionally, the 30th and 31st of December are generally given as holidays. As are January 2nd and 3rd, with January 1st being an official holiday.

I did my research and found holidays and thought great. Let's see what we can do.

Findings

The package 'holiday' provides lots of nice features and accurately returns dates as holidays.

Let's take the recent Golden Week. For 2025, みどりの日, (Greenery Day) falls on Sunday May 4th. With みどりの日 being moved to the 7th since it falls on a Sunday and Monday the 6th being 子どもの日, Children's Day. This all seems fine. But I have a slight issue. I need Sunday to be returned as a non national holiday. As that day is not observed as a national holiday but is a normal weekend day.

After some scratching, google searching and finally looking the the GitHub repo of holidays. I came across some documentation for creating a customised holiday object.

The Solution that works for me

This is the basic solution that I came up.

It follows the basic example in the packages own documentation.

Line 17 pulls in the corporate holidays
Lines 18 to 22 add the additional holidays that I require
Line 23 pulls in the National Holidays for Japan
Lines 24 to 27 then remove any National Holiday that falls on a Sunday

Conclusion

This worked out really well. So if you need to do something that requires custom national and corporate holidays. I suggest you give this package a spin.

A barebones C++ library template using CMake and Google Test

Adam.S — Fri, 10 Dec 2021 07:53:06 +0000

Objective

This is a BAREBONES C++ template with Google Test support using CMake. It is setup to allow for the building of a static library using TDD principles; but could be adapted for building a small program. This is in no way the ideal solution.

This template was created to allow for programming under Mac OS. But the files should work under other systems.

Directory Structure

This is the structure that we start with. It should be noted that the build directory will be generated in the root of this structure under VSCode. This appears, to generally be discouraged and can be changed with some configuration options that I will not go into here.

❯ tree .
.
├── CMakeLists.txt
├── README.md
├── src
│   ├── CMakeLists.txt
│   ├── Example.cpp
│   └── Example.hpp
└── tests
    ├── CMakeLists.txt
    └── lib-tester.cpp

Requirements

You will require the following items to be installed. Note that I have used homebrew in order to source the requirements. If you are not using Mac OS you should be able to find other ways to install the requirements. Google Search will be your friend in this.

CMake

Installation

brew install cmake

Google Test

Installation

brew install googletest

Example Execution

Let's build the code in and Out of Source build folder.

❯ cmake -B /tmp/testing -S .
-- The CXX compiler identification is AppleClang 13.0.0.13000029
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found GTest: /usr/local/lib/cmake/GTest/GTestConfig.cmake (found version "1.11.0")  
-- Configuring done
-- Generating done
-- Build files have been written to: /tmp/testing

Let's compile the code

❯ cd /tmp/testing
❯ ls
CMakeCache.txt      CTestTestfile.cmake cmake_install.cmake tests
CMakeFiles          Makefile            src
❯ make
[ 25%] Building CXX object src/CMakeFiles/mylib.dir/EXample.cpp.o
[ 50%] Linking CXX static library libmylib.a
[ 50%] Built target mylib
[ 75%] Building CXX object tests/CMakeFiles/Test1Tester.dir/lib-tester.cpp.o
[100%] Linking CXX executable Test1Tester
[100%] Built target Test1Tester

Things to note: The generated static library will be called libmylib.a and its name comes from the project variable in the top level CMakeList.txt file.

CMakeList.txt Contents

Root

cmake_minimum_required(VERSION 3.22)
project(mylib
        VERSION 0.1.0.0
        DESCRIPTION "What does it do?"
        LANGUAGES CXX
        )

set(CMAKE_CXX_STANDARD 14)
set(CMAKE_CXX_STANDARD_REQUIRED true)
set(CMAKE_POSTION_INDEPENDENT_CODE ON)

enable_testing()

add_subdirectory(src)
add_subdirectory(tests)

The project label here is being used to set several variables.

mylib This is used to set the library's name -> libmylib.a
enable_testing() This is used to enable to testing within the IDE
add_subdirectory() This makes the top level aware of the sub-directories. src is the code source of the library. And tests is where the source code for the testing code will be written.

src

set(LIB_BINARY ${CMAKE_PROJECT_NAME})

set(headers
    Example.hpp)
set(sources
    EXample.cpp)
set(SOURCES ${headers} ${sources})

add_library(${LIB_BINARY} STATIC ${SOURCES})
target_include_directories( ${LIB_BINARY} PUBLIC ${CMAKE_CURRENT_SOURCE_DIR})

LIB_BINARY This stores the name of the library to be generated. Coming from project which was set in the previous CMakeList.txt covered earlier.
headers This will contain references to any and all header files that are need to compile the library.
sources This will contain references to any source files that are needed.
SOURCES This combines headers and sources
add_library This tells cmake that we are declaring we want to generate a library file and not an executable. We also specifiy that it is to be a Static and not dynamic library.
target_include_directories this tells cmake that we should:
- build a library named lib{LIB_BINARY}.a
- This directory's contents should be made available to other directories in the main project. This means we will be able to build the test programs using this new library.

tests

cmake_minimum_required(VERSION 3.22)

set(CMAKE_CXX_STANDARD 14)
set(CMAKE_CXX_STANDARD_REQUIRED true)
set(CMAKE_POSTION_INDEPENDENT_CODE ON)

file(GLOB_RECURSE TEST_SOURCES LIST_DIRECTORIES false *.hpp *.cpp)

set(test1 Test1Tester)
set(test1src lib-tester.cpp)

# Specify second test program and source files
#set(test2 Test2Tester)
#set(test2src lib-tester2.cpp)

add_executable(${test1} ${test1src})

# Specify executable for second tester
#add_executable(${test2} ${test2src})

find_package( GTest REQUIRED )      
# adds the gtest include directory
include_directories( ${GTEST_INCLUDE_DIRS} )
# links gtest
target_link_libraries( ${test1} LINK_PUBLIC ${GTEST_LIBRARIES} ${CMAKE_PROJECT_NAME})

# Specify target for second test program
#target_link_libraries( ${test2} LINK_PUBLIC ${GTEST_LIBRARIES} ${CMAKE_PROJECT_NAME})

enable_testing()
add_test(
   NAME ${test1}}
   COMMAND ${test1}
)

file This another way to get gather the files for compiling. Though it is discouraged.
test1 This will hold the name of the testing executable.
test1src This holds the name of the source file to be used to build the test executable.
add_executable This is the executable to be compiled as the test program

We also need to ensure that we can link in the GTest library. We do that with the following commands.

find_package( GTest REQUIRED )      
# adds the gtest include directory
include_directories( ${GTEST_INCLUDE_DIRS} )

This locates the Google Test libraries that were installed earlier.

Linking the everything together so we have an executable that tests our code.

target_link_libraries( ${test1} LINK_PUBLIC ${GTEST_LIBRARIES} ${CMAKE_PROJECT_NAME})

This creates the executable, linking the GTEST library and the mylib library

If you look through you will see that I have commented out a second source and executable. It is possible to build several test programs. This allows you to build different tests programs instead of having one single monalithic test application.

Screen Capture

Github Repo

You can get the source here

Moving Site from GitHub

Adam.S — Sat, 03 Jul 2021 13:07:46 +0000

Hi All,

So yesterday I decided to move my site from github.io to one of my personal domains. I am not sure why I decided this. I mean Github provides a great free service. And I had lots of automation in place. In this article I will just cover how i went about updating all pages with redirects from github to the new domain.

The Goal

My goal was to leave my old site up on GitHub for a few weeks to give Google some time to update its indexing. With all old pages being able to redirect to their new counterparts. This requires that I have redirects in place for each page. I will then remove the github site repo.

Limitations with Github

From my understanding. There is no support for server side redirects. This left me with the option of using HTML/JavaScript redirects.

Resources and Research

I did a fair amount of reading before tacking this mini project.

Implementation

I chose to go with the HTML rewrite approach.

One of the reason for this is that I create my site using two Github repos. One is private with some workflow actions which populates the second public repo. The public repo only has the generated webpage code.

Initial Setup

Since I am focusing on the re-write script. I am going to assume you have done the following things.

Have the new site up and running on the new domain.
Disabled Google Analytics and AdSense on the old site, or already moved these properties to the new domain if you are using them.

Getting things into place.

In my case this was pretty straight forward. My site is being served from a repo that only contains the page code. So working in a seperate area from my normal local copy of my private repo I simply cloned the public site to my local machine.

git clone git@github.com:Bas-Man/bas-man.github.io.git

Within this directory. I then removed files and folders that would not be needed.

css
images
fonts

Those sorts of things.

The script components

New HTML file contents

I decided to go with the following html code with some adjustments.

<!DOCTYPE HTML>                                                                 
<html lang="en">                                                                
    <head>                                                                      
        <meta charset="utf-8">
        <meta http-equiv="refresh" content="0;url={{THE_NEW_URL}}" />       
        <link rel="canonical" href="{{THE_NEW_URL}}" />                     
    </head>                                                                                                                                                                   
    <body>                                                                      
        <h1>                                                                    
            The page been moved to <a href="{{THE_NEW_URL}}">{{THE_NEW_URL}}</a>
        </h1>                                                                   
    </body>                                                                     
</html>

Within this local repo I created a file called source with the following contents

<!DOCTYPE HTML>                                                                 
<html lang="en">                                                                
    <head>                                                                      
        <meta charset="utf-8">
        <meta http-equiv="refresh" content="0;url=https://bas-man.dev/NAME" />       
        <link rel="canonical" href="https://bas-man.dev/NAME" />                     
    </head>                                                                                                                                                                   
    <body>                                                                      
        <h1>                                                                    
            The page been moved to <a href="https://bas-man.dev/NAME">https://bas-man.dev</a>
        </h1>                                                                   
    </body>                                                                     
</html>

The value NAME is just a place holder and will be replaced in each file using the output of the good old trusty sed command.

Transversing the Directory Structure.

For this I chose the following.

find . -print0 | while IFS= read -r -d '' file
do 
    echo "$file"
done

And did some testing which gave a huge amount of output.

--snip--
./categories/coding/page/4/index.html
./categories/coding/page/3
./categories/coding/page/3/index.html
./categories/coding/page/2
./categories/coding/page/2/index.html
./categories/coding/page/5
./categories/coding/page/5/index.html
./categories/coding/index.html
./categories/automation
./categories/automation/page
./categories/automation/page/1
./categories/automation/page/1/index.html
./categories/automation/index.html
./categories/front-end
./categories/front-end/page

Straight away I noticed there would be an issue with the ./ at the start of each line. But I will get to that.

What do I actually want to do

What I want to do is to replace any file ending in .html with the contents of source where name has been replaced with the path to the html file. Let's take a look at a simple example from above

./categories/coding/index.html needs to have its contents replaced with the contents of source but with NAME replaced with categories/coding/index.html.

Example

<!DOCTYPE HTML>                                                                 
<html lang="en">                                                                
    <head>                                                                      
        <meta charset="utf-8">
        <meta http-equiv="refresh" content="0;url=https://bas-man.dev/categories/coding/index.html" />       
        <link rel="canonical" href="https://bas-man.dev/categories/coding/index.html" />                     
    </head>                                                                                                                                                                   
    <body>                                                                      
        <h1>                                                                    
            The page been moved to <a href="https://bas-man.dev/categories/coding/index.html">https://bas-man.dev</a>
        </h1>                                                                   
    </body>                                                                     
</html>

The final Script and Break Down

find . -print0 | while IFS= read -r -d '' file
do 
    if [ "${file: -4}" == "html" ] ; then
       cat ./source | sed "s|NAME|${file:2}|g" > ${file}
    fi
done

The if statement

This check if the file ends in html. To do this we use ${file: -4} which return the last 4 characters of file. If this is true we will replace the contents of file. ##### Re-write the file contents
Replacing the contents is done using old classic tools. cat, sed and > the output redirect.
- I pass the contents of source to sed.
- Sed then replaces NAME with the value of file starting from the 3rd character. We count from 0. So 0,1,2 <- 3 position.
- I use double quotes since I am doing interpolation
- I use | since of / as my separator since my string also contains / characters.
- I use the g option since I need to place all occurances of NAME, so using greedy mode.

With this done. I take the easy way and run the script by a shell. The script was simply called update

sh update

And all the HTML files are updated with the correct references to the exact same pages, but to the new domain.

Update the old site

From here it was a simple matter to update the repo with some git commands and push the updates back up to GitHub

git add .
git reset -- update
git reset -- source
git commit -m "my comment"
git push -u origin

The resets were just to remove the two files I didn't need pushed to the live site.

Closing

That's about it. Seems to have worked pretty well. I just need to keep an eyes on things and see if I need to make any changes.

Hope this might help others.

Push to GitHub Triggers Push to Remote

Adam.S — Tue, 29 Jun 2021 04:55:31 +0000

Hi,

Today I would like to go over how I changed things up a bit and am now able to push my site updates to GitHub, which then pushes the updates to the live production server. This should allow me to add checks within GitHub to ensure my site it updated only if everything is correct. Though I won't go into that here today. I will be focusing on the GitHub Action.

The Idea

Again the basic idea is pretty simple. I will push updates to my GitHub repo. This will then cause GitHub to trigger an action that will push the updates to my live production site server. In this case I will be using the actions/checkout instead of pushing directly to the production server. Less risk of mistakes, if you have checks in place on GitHub.

Requirements

Actions must be done by unprivileged users.
Connect over SSH

Getting Everything we Need

SSH

We need to generate a new ssh key pair. For simplicity I went with the standard rsa with 4096.

ssh-keygen -t rsa -b 4096 -C "[identifier]@linode" -f linode

This creates two files. linode the private key. The one we need to keep really safe. And linode.pub the public key that we usually share.

Things are going to be a little different here since we are actually going to put the private key into GitHub as a repository secret. And add the public key to the remote, linode, server.

Before we do that we also need to get some information about the destination server. We need the information that would normally be stored in the known_hosts file when we connect to a remote server using standard ssh

We can get this information with:

ssh-keyscan -t rsa [host address/IP address]

You should see that I used -t rsa here because my ssh keys are also rsa.

Add these ssh details to our GitHub repo. We have to open our repo on GitHub and add our secrets to the repo we will be pushing to.

Access Settings -> Secrets and create the following two secrets. Using the New Respository Secret button.

Name: LINODE_KEY

Contents:

-----BEGIN OPENSSH PRIVATE KEY-----
b3BlbnNzaC1rZXktdjEAAAAABG5vbmUAAAAEbm9uZQAAAAAAAAABAAACFwAAAAdzc2gtcn
--snip--
----------END OPENSSH PRIVATE KEY-----

I know this looks odd but, Github will keep this safe. And I would use a unique key anyway. Just in case you need to burn it in the future.

Next we need to add the second secret.

Name: LINODE_KNOWN_HOSTS

Contents: This will be the output you get from the ssh-keyscan command listed earlier.

We next need to add the contents of linode.pub to the remote server's authorized_keys under .ssh for the user we plan to connect as. There are lots of guides online for this.

GitHub Action

At this point we can start putting the GitHub Action together. We do this under .github/workflows/

name: Linode Push Update

on:
  push:
    branches:
      - master

jobs:
  build:
    name: Build
    runs-on: ubuntu-latest
    steps:
    - name: Prepare ssh keys
      uses: shimataro/ssh-key-action@v2
      with:
        key: ${{ secrets.LINODE_KEY }}
        known_hosts: ${{ secrets.LINODE_KNOWN_HOSTS }}

    - name: Checkout the Repo
      uses: actions/checkout@v2
      with:
        ssh-key: ${{ secrets.LINODE_KEY }}
        fetch-depth: 0
        ref: master
    - name: Push to Linode
      run: |
        git config user.name github-actions
        git config user.email github-actions@github.com
        git remote add prod ssh://[username]@[host]:/var/repos/[remote-repo-name].git
        git push prod

Most of this is pretty straight forward. But I was missing an important step initially. That is the use to of the shimataro/ssh-key-action. This action is needed as it will add our secret ssh data so that we authenticate when we connect to the remote server. It adds the LINODE_KEY and the LINODE_KNOWN_HOSTS information.

If you are wondering about the remote-repo-name.git this is because it should be created as a bare git repo. If you want to know more about how I created it and the post-receive script that is eventually triggered. Take a look at my previous post here

Using this approach I was able to reuse my previously written post-receieve hook script on the remote server to update the live site. So I didn't need to change anything on the remote server.

Closing

I hope that was interesting and helps others.

Auto Update Site from a Git Push

Adam.S — Thu, 24 Jun 2021 21:54:21 +0000

Hi,

For a little while now I have been thinking about moving my personal blog to a self-hosted service. One of the options I have been considering is Linode.

As part of this idea. I wanted to see how I might be able to automate some of the processes. This is just a write up on one of my ideas so far.

Idea

The basic idea is pretty simple. After I have created and edited a post. I will push the updates to the remote hosting server. It will then rebuild the site on the hosting server and update the site. At the moment I use Hugo as my static site generator. As part of that I am looking at using a docker image with let's Encrypt and Nginx. I may cover that in a future post.

requirements

Actions must be done by unprivileged users.
rsync is installed
git is installed
hugo is installed (This would change if you were using a different site generator)

Remote Server Environment

I am going to assume you have an existing unprivileged user. So I will not cover that here. If you need help. Google can be your friend.

The remote server will have two main directories.

Bare git Repo to accept the push from my local development environment.
Normal directly from which the site will be served by the chosen web server.

In my case I chose to create /var/www/ and var/repos. These directories are owned by root. Below these I created the actual folders that git and the web server will use.

WWW

Here I created a folder using the name of my domain.

sudo mkdir -p /var/www/[domain-name]
sudo chown [myuser]:[myuser] /var/www/[domain-name]
chmod 755 /var/www/[domain-name]

The last two commands change the owner of domain-name so that my unprivileged user will have access to this directory. Only the owner will have write access. But anyone can read. This is important as the web server will probably be a different user.

Repos

Here we create the directory for storing git repos. And specifically the repo directory for the site. We also make the directory a special git directory.

sudo mkdir -p /var/repos/[domain-name].git
sudo chown [myuser]:[myuser] /var/repos/[domain-name].git
cd /var/repos/[domain-name].git
git --bare init

post-receive hook

We need to create a post-receive hook that will be triggered when we receive a push from our development server.

#!/bin/bash

DOMAIN=[domain-name]
GIT_REPO=/var/repos/[domain-name].git
WORKING_DIRECTORY=/tmp/$DOMAIN
PUBLIC_WWW=/var/www/$DOMAIN
BACKUP_WWW=/tmp/$DOMAIN-backup

set -e

rm -rf $WORKING_DIRECTORY
rsync -aqz $PUBLIC_WWW/ $BACKUP_WWW
trap "echo 'A problem occurred.  Reverting to backup.'; rsync -aqz --del $BACKUP_WWW/ $PUBLIC_WWW; rm -rf $WORKING_DIRECTORY" EXIT

git clone $GIT_REPO $WORKING_DIRECTORY
# Add any git submodules here so that they are included during the build.
git clone https://github.com/[my sites theme] $WORKING_DIRECTORY/themes/[my theme name]
rm -rf $PUBLIC_WWW/*
hugo -s $WORKING_DIRECTORY -d $PUBLIC_WWW -b "http://${DOMAIN}"
rm -rf $WORKING_DIRECTORY
trap - EXIT

Footnote¹
This script basically clones the default branch into a /tmp/ directory. Collects the required theme module and then has hugo build the site directly into the live site folder. There is an issue that if you remove any content from your git repo. It will not be removed from the live site with this approach.

Note
This script could be improved by building the site in another temp directory, and then using rsync to copy this into the live site directory whist deleting any missing files and or folders. Though I did not create this script myself.

Local Development Environment

This part is pretty straight forward. We are simply going to add a new remote repo.

Inside our development git repo we can simply issue the command:

git remote add prod [myuser]@[host]:/var/repos/[domain-name].git

This will allow us to push our update directly to the live web server just by using the command

git push prod

But before we can do this we need to setup our ssh key access.

SSH Access

Regarding SSH. There are many great guides online for this. So I will just suggest that you take a look around.

This script and idea was taken from Digital Ocean ↩

Rust: Add an Example Program to your Library Crate

Adam.S — Mon, 21 Jun 2021 00:43:48 +0000

Recently I decided that I might actually publish my little rust crate. To that end I started looking into some of the things that are needed and what can be done. I have published a perl module in the past. And as a rule for myself, I like to provide an example program that uses a module or library. That is what I will look at today.

When I started this project to learn rust, I was simply making a single program. As things evolved, it became more complex and turned in to a program that uses a library. This was all fine.

But as I looked at making this a library crate. I noticed when I compiled my library it was also bringing in external crates that my library didn't need. Why? Because my crate was a library and an binary.

Cargo.toml

Old

[package]
-- snip --
edition = "2018"

[dependencies]
trust-dns-resolver = "0.20.1"
ipnetwork = "0.17.0"
regex = "1"

In the old toml file we can see that all of my dependencies are listed together. Even though my library only really makes use of ipnetwork and regex. At this point my regex was not using the lazy_static crate.

I also have not listed any [lib] section.

New

[lib]
name = "redacted" (at least until I publish it)
path = "src/lib.rs"

[[example]]
name = "trust-dns-demo"
path = "examples/trust-dns-resolver.rs"

[dependencies]
ipnetwork = "0.17.0"
regex = "1"
lazy_static = "1.4.0"

[dev-dependencies]
trust-dns-resolver = "0.20.1"

In this new toml file. I have moved the trust-dns-resolver down under [dev-dependencies]. In this way when I compile my library the trust-* crate and any of its dependencies will not be compiled. This has an added benefit that when docs are created. Only items under [dependencies] are included in the docs.

Though you can also run:

cargo doc --no-deps

This will prevent docs from generating docs for dependencies.

Crate Directory Structure

Old

├── Cargo.lock
├── Cargo.toml
├── LICENSE
├── README.md
└── src
    ├── lib.rs
    ├── main.rs
    └── spf

The old setup has a main.rs. This in effect tells cargo that this a binary crate. The [lib] in Cargo.toml also tells it that it is library crate.

The affect of having a main.rs is that both a binary and library are compiled. In my case main.rs has a dependency on trust-dns-resolver.

To make this compile as only a library and also provide an executable example program, the solution is to define an example target.

As seen above.

[[example]]
name = "trust-dns-demo"
path = "examples/trust-dns-resolver.rs"

This defines a new target that can be compiled. Notes from The Cargo Book specifically,
Example states.

Files located under the examples directory are example uses of the functionality provided by the library. When compiled, they are placed in the target/debug/examples directory.

Examples can use the public API of the package's library. They are also linked with the [dependencies] and [dev-dependencies] defined in Cargo.toml.

New

.
├── Cargo.lock
├── Cargo.toml
├── LICENSE
├── README.md
├── examples
│   └── trust-dns-resolver.rs
└── src
    ├── lib.rs
    └── spf

What I did here was to create an examples directory, rename and move the main.rs to this new directory as trust-dns-demo.rs

The result

When I run a build. I only use the main dependencies.

$ cargo build
  Compiling memchr v2.3.4
  Compiling serde v1.0.125
  Compiling regex-syntax v0.6.23
  Compiling lazy_static v1.4.0
  Compiling aho-corasick v0.7.15
  Compiling regex v1.4.6
  Compiling ipnetwork v0.17.0
  Compiling something v0.1.0 (/Users/me/Documents/Development/rust/something/

However, when I run a test.

cargo test
   Compiling libc v0.2.96
   Compiling cfg-if v1.0.0
   Compiling proc-macro2 v1.0.27
   Compiling unicode-xid v0.2.2
   -- snip 40 items for brevity --
   Compiling idna v0.2.3
   Compiling rand v0.8.3
   Compiling url v2.2.2
   Compiling thiserror-impl v1.0.25
   Compiling enum-as-inner v0.3.3
   Compiling thiserror v1.0.25
   Compiling trust-dns-proto v0.20.3
   Compiling trust-dns-resolver v0.20.3
   Compiling something v0.1.0 (/Users/me/Documents/Development/rust/something)
    Finished test [unoptimized + debuginfo] target(s) in 21.68s
     Running unittests (target/debug/deps/something-49ff56fbf5edcf40)

running 69 tests
test spf::kinds::test_kind_ip4 ... ok
test spf::kinds::test_kind_exists ... ok
test spf::kinds::test_kind_a ... ok
-- Snip 60 items for brevity --
test spf::tests::spf::test_spf::test_redirect ... ok
test spf::tests::spf::test_spf::test_netblocks2_google_com ... ok
test spf::tests::spf::ptr::capture::test_match_on_ptr ... ok
test spf::tests::spf::ptr::parse::test_exist ... ok
test spf::tests::spf::ptr::parse::test_exist_colon ... ok
test spf::tests::spf::ptr::capture::test_match_on_ptr_colon ... ok

test result: ok. 69 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.01s

   Doc-tests something

running 5 tests
test src/spf/mod.rs - spf::Spf::from_str (line 66) ... ok
test src/spf/kinds.rs - spf::kinds::MechanismKind::as_str (line 82) ... ok
test src/spf/mechanism.rs - spf::mechanism::Mechanism<IpNetwork>::raw (line 155) ... ok
test src/spf/mechanism.rs - spf::mechanism::Mechanism<String>::new_a (line 65) ... ok
test src/spf/mechanism.rs - spf::mechanism::Mechanism<IpNetwork>::string (line 171) ... ok

test result: ok. 5 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.64s

Note the inclusion of:

Compiling trust-dns-proto v0.20.3
Compiling trust-dns-resolver v0.20.3

Making the Example program Run

This can be done pretty simply.

$ cargo run --example
error: "--example" takes one argument.
Available examples:
    trust-dns-demo

So we can simply run

$ cargo run --example trust-dns-demo
Finished dev [unoptimized + debuginfo] target(s) in 0.21s
 Running `target/debug/examples/trust-dns-demo`
List of TXT records found for gmail.com.
TXT Record 1:
v=spf1 redirect=_spf.google.com
TXT Record 2:
globalsign-smime-dv=CDYX+XFHUw2wml6/Gb8+59BsH31KzUr6c1l2BPvqKX8=

Decontructing SPF Record
Spf { source: "v=spf1 redirect=_spf.google.com", version: "v=spf1", from_src: true, include: None, redirect: Some(Mechanism { kind: Redirect, qualifier: Pass, mechanism: "_spf.google.com" }), is_redirected: true, a: None, mx: None, ip4: None, ip6: None, ptr: None, exists: None, all: None }
SPF1: v=spf1 redirect=_spf.google.com


Is a redirect: true

redirect: _spf.google.com
mechanism: redirect=_spf.google.com

Closing

That's it for this article. I hope it is a fairly understandable way to provide an example program along with your rust library.

Published My First Crate: decon-spf

Adam.S — Tue, 15 Jun 2021 08:10:39 +0000

Today I published my first rust crate.
I have to say I am still pretty excited and nervous about it. I decided to release it today because I realised that if I didn't. I would keep making changes and decide it was never really ready.

I know I still have more things that I want to add to it. But the core functionality is present.

So if you are curious. Head over and take a look at decon-spf. From there you can get to the GitHub repo if you want to see more details.

Rust: Lazy Static Regex

Adam.S — Tue, 15 Jun 2021 03:24:57 +0000

Hi,

Today I will finally go over how I used the lazy_static crate to help the compiler keep regex optimised.

References

Avoid using regex in a loop

From the regex documentation it is sub-optimal and anti-pattern to compile the same regular expression in a loop.

Not only is compilation itself expensive, but this also prevents optimizations that reuse allocations internally to the matching engines.

Initial Code

My original code was of course the anti-pattern.

pub fn parse(&mut self) {
    // initialises required variables.
    let records = self.source.split_whitespace();
    let mut vec_of_includes: Vec<SpfMechanism<String>> = Vec::new();
    let mut vec_of_ip4: Vec<SpfMechanism<IpNetwork>> = Vec::new();
    let mut vec_of_ip6: Vec<SpfMechanism<IpNetwork>> = Vec::new();
    let mut vec_of_a: Vec<SpfMechanism<String>> = Vec::new();
    let mut vec_of_mx: Vec<SpfMechanism<String>> = Vec::new();
    for record in records {
        // Make this lazy.
        let a_pattern =
            Regex::new(r"^(?P<qualifier>[+?~-])?(?P<mechanism>a(?:[:/]{0,1}.+)?)").unwrap();
        let mx_pattern =
            Regex::new(r"^(?P<qualifier>[+?~-])?(?P<mechanism>mx(?:[:/]{0,1}.+)?)").unwrap();
        if record.contains("redirect=") {
            // Match a redirect
        --snip--
      }

This actually caused a significant issue.

To reuse these regular expressions. I had to redeclare them in all files where these were used. Test files and the like. This of course introduces increased complexity for management and thus potential mistakes.

Remove duplication

The first step was to make them const &str types.

const MECHANISM_A_PATTERN: &str = r"^(?P<qualifier>[+?~-])?a(?P<mechanism>[:/]{0,1}.+)?";
const MECHANISM_MX_PATTERN: &str = r"^(?P<qualifier>[+?~-])?mx(?P<mechanism>[:/]{0,1}.+)?";
const MECHANISM_PTR_PATTERN: &str = r"^(?P<qualifier>[+?~-])?ptr(?P<mechanism>[:]{0,1}.+)?";

This allowed me to have single place to edit the expressions if needed.

let a_pattern = Regex::new(MECHANISM_A_PATTERN).unwrap();
let mx_pattern = Regex::new(MECHANISM_MX_PATTERN).unwrap();
let ptr_pattern = Regex::new(MECHANISM_PTR_PATTERN).unwrap();

But this still meant I need to include the patterns with a use statement in each file.

In a test file for example.


use crate::spf::helpers;
use crate::spf::MECHANISM_A_PATTERN;
use regex::Regex;

#[test]
fn test_match_on_a_only() {
    let string = "a";
    let pattern = Regex::new(MECHANISM_A_PATTERN).unwrap();
    let option_test: Option<Mechanism<String>>;

    option_test = helpers::capture_matches(pattern, &string, kinds::MechanismKind::A);

    let test = option_test.unwrap();
    assert_eq!(test.is_pass(), true);
    assert_eq!(test.raw(), "a");
    assert_eq!(test.string(), "a");
}

At this point you might notice I have a helpers::. I needed to use this function a lot in my tests files. So I moved it to the helpers.rs file.

helpers.rs

Let's take a look.

use crate::spf::kinds;
use crate::spf::mechanism::Mechanism;
use crate::spf::qualifier::Qualifier;
use regex::Regex;

#[doc(hidden)]
pub(crate) fn capture_matches(
    pattern: Regex,
    string: &str,
    kind: kinds::MechanismKind,
) -> Option<Mechanism<String>> {
--snip--
}

Here I am using pub(crate) which defines that this function is public within the crate and only within the crate.

This still does not get me out of the inside loops compile / optimisation issue.

The new helper.rs

To make regex expressions truely lazy and so the compiler only does its work once. These regex need to be created within a helper function. Fortunately, I basically already had one in the form of capture_matches().

Let's revisit helpers.rs

use lazy_static::lazy_static;
use regex::Regex;

pub(crate) const MECHANISM_A_PATTERN: &str = r"^(?P<qualifier>[+?~-])?a(?P<mechanism>[:/]{0,1}.+)?";
pub(crate) const MECHANISM_MX_PATTERN: &str =
    r"^(?P<qualifier>[+?~-])?mx(?P<mechanism>[:/]{0,1}.+)?";
pub(crate) const MECHANISM_PTR_PATTERN: &str =
    r"^(?P<qualifier>[+?~-])?ptr(?P<mechanism>[:]{0,1}.+)?";

pub(crate) fn capture_matches(
    string: &str,
    kind: kinds::MechanismKind,
) -> Option<Mechanism<String>> {
    lazy_static! {
        static ref A_RE: Regex = Regex::new(MECHANISM_A_PATTERN).unwrap();
        static ref MX_RE: Regex = Regex::new(MECHANISM_MX_PATTERN).unwrap();
        static ref PTR_RE: Regex = Regex::new(MECHANISM_PTR_PATTERN).unwrap();
    }
    let caps = match kind {
        kinds::MechanismKind::A => A_RE.captures(string),
        kinds::MechanismKind::MX => MX_RE.captures(string),
        kinds::MechanismKind::Ptr => PTR_RE.captures(string),
        _ => unreachable!(),
    };
    let qualifier_char: char;
    let mut qualifier_result: Qualifier = Qualifier::Pass;
    let mechanism_string: String;
    let mechanism;
    match caps {
        None => return None,
        Some(caps) => {
            // There was a match
            if caps.name("qualifier").is_some() {
                qualifier_char = caps
                    .name("qualifier")
                    .unwrap()
                    .as_str()
                    .chars()
                    .nth(0)
                    .unwrap();
                qualifier_result = char_to_qualifier(qualifier_char);
            };
            if caps.name("mechanism").is_some() {
                mechanism_string = caps.name("mechanism").unwrap().as_str().to_string();
                mechanism = Mechanism::new(kind, qualifier_result, (*mechanism_string).to_string());
            } else {
                mechanism_string = match kind {
                    kinds::MechanismKind::A => "a".to_string(),
                    kinds::MechanismKind::MX => "mx".to_string(),
                    kinds::MechanismKind::Ptr => "ptr".to_string(),
                    _ => unreachable!(),
                };
                mechanism = Mechanism::new(kind, qualifier_result,
                    mechanism_string);
            }

            Some(mechanism)
        }
    }
}

Here the const &str have also been made public to the crate.

I am then creating A_RE, MX_RE and PTR_RE within this function enclosed within a lazy_static macro block. This means that the regular expressions are now part of the function and will be used any time the function is called. So I was able to remove the pattern: Regex from the parameter list.

My case may or may not be unusual in that I am using the same function for three different patterns.

I handle this by passing kinds::MechanismKind::{A,MX,PTR}. I use match to then ensure that the correct regular expression is applied to the string passed in the first parameter. The _ => unreachable!() macro will cause the program to panic since this branch should never be reached. The _ tells rust that we are not interested in other possible arms of the match case.

The new Code

Now we can use this code more easily and keep the code much cleaner.

Another gain is that we no longer need to use regex in other files. We now just need to call use crate::spf::helpers; and we have access to our capture_matches() function.

parse()

This function no longer declares and regular expressions.

pub fn parse(&mut self) {
    // # TODO: This needs to have a test for emtpy source. return a Result?
    // initialises required variables.
    let records = self.source.split_whitespace();
    let mut vec_of_includes: Vec<Mechanism<String>> = Vec::new();
    let mut vec_of_ip4: Vec<Mechanism<IpNetwork>> = Vec::new();
    let mut vec_of_ip6: Vec<Mechanism<IpNetwork>> = Vec::new();
    let mut vec_of_a: Vec<Mechanism<String>> = Vec::new();
    let mut vec_of_mx: Vec<Mechanism<String>> = Vec::new();
    let mut vec_of_exists: Vec<Mechanism<String>> = Vec::new();
    for record in records {
        if record.contains("v=spf1") || record.starts_with("spf2.0") {
            self.version = record.to_string();
    -- snip --
        } else if let Some(a_mechanism) =
            helpers::capture_matches(record, kinds::MechanismKind::A)
        {
            vec_of_a.push(a_mechanism);
        } else if let Some(mx_mechanism) =
            helpers::capture_matches(record, kinds::MechanismKind::MX)
        {
            vec_of_mx.push(mx_mechanism);
        } else if let Some(ptr_mechanism) =
            helpers::capture_matches(record, kinds::MechanismKind::Ptr)
        {
            self.ptr = Some(ptr_mechanism);
        }
    -- snip --
}

Test File

The test file also now looks much nicer.

use crate::spf::helpers;
use crate::spf::kinds;
use crate::spf::mechanism::Mechanism;

#[test]
fn test_match_on_a_only() {
    let string = "a";
    let option_test: Option<Mechanism<String>>;

    option_test = helpers::capture_matches(&string, kinds::MechanismKind::A);

    let test = option_test.unwrap();
    assert_eq!(test.is_pass(), true);
    assert_eq!(test.raw(), "a");
    assert_eq!(test.string(), "a");
}

Final Words.

So the take away here is that if you want to make your regulgar expressions lazy, and help the compiler keep things optimised.

Create a helper function that does the pattern matching and define your lazy_static! {} within that function. Don't forget to add the lazy_static crate to your Cargo.toml.

Thoughts on Rust Builder Pattern?

Adam.S — Sun, 13 Jun 2021 09:48:35 +0000

Update with Playground Builder Code

Today I took a look at an interesting post related to JavaScript and the Builder Pattern

JavaScript Design Patterns - Builder

Coderslang: Become a Software Engineer ・ Jun 12 ・ 5 min read

#javascript #webdev #codenewbie #beginners

This got me wondering about Rust and the Builder Pattern

This lead me to some interesting articles on the topic.

I presently have no opinion myself as I am rather new. But I was hoping some others with more experience might venture their opinions. It seems people are a bit split about the use of the Builder Pattern in rust.

You can access the Playound

The Gist

I used the following sample from Rust Unoffical as a starting point.

The code if you just want to see it:

#![allow(unused)]
#[derive(Debug, Clone)]
pub struct OTG<'a> {
    model: String,
    title: &'a str,
    color: String,
    max_temperature:u8,
    max_time_selection:u8,
}

impl<'a> Default for OTG<'a> {
    fn default() -> OTG<'a> {
        OTG {
            model: String::new(),
            title: "OTG",
            color: String::new(),
            max_temperature: 150,
            max_time_selection: 30,
        }
    }

}
impl<'a, 'b> OTG<'a> {
    fn new(model: String, color: String) -> Self {
        Self {
            model,
            color,
            ..OTG::default()
        }
    }
    pub fn builder() -> OTGBuilder<'b> {
        OTGBuilder::default()
    }
}

#[derive(Default, Clone)]
pub struct OTGBuilder<'b> {
    model: String,
    title: &'b str,
    color: String,
    max_temperature:u8,
    max_time_selection: u8
}

impl<'b> OTGBuilder<'b> {
    pub fn new() -> Self {
        Self {
            model: String::new(),
            color: String::new(),
            title: "OTG",
            max_temperature: 150,
            max_time_selection: 30,
        }

    }
    pub fn model(mut self, model: String) -> OTGBuilder<'b> {
        self.model = model;
        self
    }
    pub fn color(mut self, color: String) -> OTGBuilder<'b> {
        self.color = color;
        self
    }
    pub fn set_temp(mut self, temp: u8) -> OTGBuilder<'b> {
        self.max_temperature = temp;
        self
    }
    pub fn set_time(mut self, time:u8) -> OTGBuilder<'b> {
        self.max_time_selection = time;
        self
    }
    pub fn build(self) -> OTG<'b> {
        OTG {
            model: self.model.to_string(),
            color: self.color.to_string(),
            max_temperature: self.max_temperature,
            max_time_selection: self.max_time_selection,
            ..OTG::default()
        }
    }


}
fn main() {
    let otg = OTG::new("LG".to_string(), "Red".to_string());

    println!("Model: {}, Color: {} Temp: {}, Time: {}",
    otg.model, otg.color,
    otg.max_temperature, otg.max_time_selection
    );

    let otg_builder1: OTG = OTGBuilder::new()
                            .model(String::from("Builder1"))
                            .color("Red".to_string()).build();
    println!("Model: {}, Color: {} Temp: {}, Time: {}",
    otg_builder1.model, otg_builder1.color,
    otg_builder1.max_temperature, otg_builder1.max_time_selection
    );

    let otg_builder2: OTG = OTGBuilder::new()
                            .model(String::from("Builder1"))
                            .color("Red".to_string())
                            .set_temp(80).build();
    println!("Model: {}, Color: {} Temp: {}, Time: {}",
    otg_builder2.model, otg_builder2.color,
    otg_builder2.max_temperature, otg_builder2.max_time_selection
    );

    let otg_builder3: OTG = OTGBuilder::new()
                            .model(String::from("Builder1"))
                            .color("Red".to_string())
                            .set_temp(80)
                            .set_time(50).build();
    println!("Model: {}, Color: {} Temp: {}, Time: {}",
    otg_builder3.model, otg_builder3.color,
    otg_builder3.max_temperature, otg_builder3.max_time_selection
    );

    let otg_builder4: OTG = OTGBuilder::new()
                            .model(String::from("Builder1"))
                            .color("Red".to_string())
                            .set_time(50).build();
    println!("Model: {}, Color: {} Temp: {}, Time: {}",
    otg_builder4.model, otg_builder4.color,
    otg_builder4.max_temperature, otg_builder4.max_time_selection
    );
}

Looking forward to any comments and or responses.

Hugo Custom Info Boxes

Adam.S — Mon, 07 Jun 2021 23:04:15 +0000

The other day I got my regular summary from GoHugo a discourse site where I am a member. This lead me to an interesting blog, which then lead me to an article on how to add custom info boxes to your site if it is not provided by your theme.

links

Hugo Custom Info Boxes
- Within this post you will find the instructions to add info boxes.
Hugo Module for Info Notices

Note:
Some of the links no longer appear to work. Or have expired certificates.

I also had to add the following to my config.toml file. For either of the solutions to work.

[markup.goldmark.renderer]
  unsafe = true

Anyway. I thought I would share this in case anyone else was looking for similar.

Swift: Deconstruct SPF: Documentation

Adam.S — Fri, 04 Jun 2021 08:14:48 +0000

Hi,

I have been working on this project on and off for a bit now. So I thought it was time to go over how we might be able to not only internally document our code, but also to document it at the package level.

References

Some basics

Code commenting

// denotes a single line code comment.

Package level

/// Single line Package comment.

/**
Multi-line Comments.
Are written subString
*/

I would refer you to the sarunw link above for a good and far more in depth write up.

Getting to the documentation with in Xcode

This can simply be done by hovering your mouse over the function declaration and clicking while holding down the ⌥(Option Key)

Example

This produces the following popup.

Generating Documentation that can be hosted.

There are a few options for this.

I chose to go with Swift-doc since it can be installed using homebrew.

At present I do not need to host any documentation. Therefore I will not be going through that process here today.

Installation

$ brew install swiftdocorg/formulae/swift-doc

Build the Documentation

From the top level of your project run the following command.

swift doc generate ./Sources/ --module-name <My Module Name> --format html

If you have no public declarations. You can run this modified command.

swift doc generate ./Sources/ --module-name <My Module Name> --format html --minimum-access-level internal

This will generate your html pages in the directory .build/documentation in your projects parent directory.

Viewing the pages.

If you have Python2 installed, using a terminal. Navigate to your projects parent directory.

cd .build/documentation
python -m SimpleHTTPServer 8000

If you are looking for other options, see the link above Run a Local Server for some ideas.

Then simple open your browser at localhost:8000

Main page

Mechanism Page

GitHub

Swift-Doc already provides a Github Action that you can add to your repo. So go check it out in the links above.

Closing

Thanks for reading.

Swift: Deconstruct SPF: asMechanism with Qualifier

Adam.S — Mon, 31 May 2021 07:15:48 +0000

Hi,

In this article, I will walk through how I re-implemented the enum for Qualifier and MechanismKind. I will also deliver a working asMechanism() function. The diff is here

Enum Qualifier and MechanismKind

I reworked these to make them more self documenting and to also simplify the get function. This function could be refactored out.

Code

enum Qualifier: String {
    case Pass = "+";
    case Fail = "-";
    case SoftFail = "~";
    case Neutral = "?";
    case None = "";

    func get() -> String {
        return self.rawValue;
    }
}
enum MechanismKind: String {
    case Redirect = "redirect=";
    case Include = "include:"
    case A = "a";
    case MX = "mx";
    case Ip4 = "ip4:";
    case Ip6 = "ip6:"
    case All = "all";

    func get() -> String {
        return self.rawValue;
    }
}

Here we can see I have provided each enum case with a new rawValue and that the get function has been greatly simplified as there is no need for a switch statement. These raw values will be used by asMechanism to build the string which it returns.

asMechanism

The asMechanism function now has the ability to access the rawValue of kind and qualifier through their member functions get.

Code

func asMechanism() -> String {
    var mechanismString = String();
    mechanismString += self.qualifier.get();
    // Access the string for this mechanism's kind.
    mechanismString += self.kind.get();
    // Access the string for the mechanism
    mechanismString += self.mechanism;
    return mechanismString;
}

Testing asMechanism

I will list a few examples here.

func testAsMechanismIp4() {
    let Mech = Mechanism(k: MechanismKind.Ip4, q: Qualifier.None, m: "192.168.1.0/24");
    XCTAssertEqual(Mech.asMechanism(), "ip4:192.168.1.0/24");
}
func testAsMechanismIp4Pass() {
    let Mech = Mechanism(k: MechanismKind.Ip4, q: Qualifier.Pass, m: "192.168.1.0/24");
    XCTAssertEqual(Mech.asMechanism(), "+ip4:192.168.1.0/24");
}

func testAsMechanismIp6() {
    let Mech = Mechanism(k: MechanismKind.Ip6, q: Qualifier.None, m: "X:X:X:X/16");
    XCTAssertEqual(Mech.asMechanism(), "ip6:X:X:X:X/16");
}
func testAsMechanismRedirect() {
    let Mech = Mechanism(k: MechanismKind.Redirect, q: Qualifier.None, m: "_spf.example.com");
    XCTAssertEqual(Mech.asMechanism(), "redirect=_spf.example.com");
}
func testAsMechanismInclude() {
    let Mech = Mechanism(k: MechanismKind.Include, q: Qualifier.None, m: "_spf1.example.com");
    XCTAssertEqual(Mech.asMechanism(), "include:_spf1.example.com");
}
func testAsMechanismA() {
    let Mech = Mechanism(k: MechanismKind.A, q: Qualifier.None, m: "");
    XCTAssertEqual(Mech.asMechanism(), "a");
}
func testAsMechanismMx() {
    let Mech = Mechanism(k: MechanismKind.MX, q: Qualifier.None, m: "");
    XCTAssertEqual(Mech.asMechanism(), "mx");
}

Here is an example output within Xcode

The Green check marks denote passing tests.

The code for this project can be found here.

This is the diff between Initial-asMechanism and QualifierSuppportAsMechanism.