DEV Community: Joe Neville

Rust reqwest and Aruba AOS-CX API

Joe Neville — Fri, 12 Aug 2022 19:05:00 +0000

AOS-CX rust reqwest client

These code snippets provide an example of API calls to the Aruba AOS-CX API using Rust reqwests.

1. Dependencies

First we need to add our dependencies to the project.
To do this, update the cargo.toml file with the following:

[dependencies]
reqwest = { version = "0.11", features = ["json", "cookies"] }
tokio = { version = "1", features = ["full"] }
serde = { version = "1", features = ["derive"] }
serde_json = "1.0"

2. Login/logout

Any interaction with the AOS-CX API requires a login, and, for good house-keeping, a logout.
The API uses a username/password for authentication.

With this snippet we:

Send a Login API call
Check the request status code
Print an error to screen if the login is unsuccessful.
Send a logout call
Check the logout call status code and print to screen if this is not 200.

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let ip_add: &str = "your-ip-address";
    let full_url = format!("https://{}/rest/latest/", ip_add);

    let client = reqwest::Client::builder()
        .cookie_store(true)
        .danger_accept_invalid_certs(true)
        .build()?;
    let params = [("username", "your-username"), ("password", "your-password")];

    let resp = client
        .post(format!("{}login", full_url))
        .form(&params)
        .send()
        .await?;
    if resp.status() != 200 {
        println!("Login unsuccessful, code is {:#?}", resp.status());
    }
    let logout = client.post(format!("{}logout", full_url)).send().await?;
    if logout.status() != 200 {
        println!("Logout unsuccessful, code is {:#?}", logout.status());
    } else {
        println!("Logout successful");
    }

    Ok(())
}

3. GET some data

Once we have the login/logout framework, we can insert API calls to GET, POST, PUT, and DELETE data.
Here's an example that sends a GET call to query the firmware on a device.
The response is JSON, which we can deserialize, using a predefined struct to handle the response key, value pairs.

First we define the struct, this is for the key 'current_version', with the value given a type of String.
Then we use the reqwests client to define the URL for the GET and how to handle the returned JSON.
Note that the name of the struct is referenced against json in the response.

use serde::Deserialize;

#[derive(Deserialize)]
struct Image {
    current_version: String,
}

let firmware = client
        .get(format!("{}firmware", full_url))
        .send()
        .await?
        .json::<Image>()
        .await?;
    println!("{:?}", firmware.current_version);

4. Dynamically parse JSON

The example given in step 3 provides a great way to handle returned JSON, but we need to know what key,value pairs are returned from an API in order to be able to compose the struct.
When investigating an API, it is often a case of trial and error to find the data that we need.

So how can we send a GET call to receive data, without knowing what will be returned?

For this we can dynamically feed the JSON response into a hash-map:

First we need to bring std::collections::HashMap; into scope.
We then send the same GET call as the second example.
But for the JSON response we reference a dynamic hash-map, rather than a struct.
The downside is that we cannot call specific keys to provide data directly, the key,value pairs are now merely strings, but we can print out the whole response.

use std::collections::HashMap;

     let firmware = client
         .get(format!("{}firmware", full_url))
         .send()
         .await?
         .json::<HashMap<String, String>>()
         .await?;
     println!("{:#?}", firmware);

5. Putting it all together using a struct

Here's the full code using a struct, I've added in an extra key:

use serde::Deserialize;

#[derive(Deserialize)]
struct Image {
    current_version: String,
    booted_image: String,
}

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let ip_add: &str = "your-ip-address";
    let full_url = format!("https://{}/rest/latest/", ip_add);

    let client = reqwest::Client::builder()
        .cookie_store(true)
        .danger_accept_invalid_certs(true)
        .build()?;
    let params = [("username", "your-username"), ("password", "your-password")];

    let resp = client
        .post(format!("{}login", full_url))
        .form(&params)
        .send()
        .await?;
    if resp.status() != 200 {
        println!("Login unsuccessful, code is {:#?}", resp.status());
    }

    let firmware = client
        .get(format!("{}firmware", full_url))
        .send()
        .await?
        .json::<Image>()
        .await?;

    println!(
        "Current firmware image: {}\nBoot firmware image: {}",
        firmware.current_version, firmware.booted_image
    );
    let logout = client.post(format!("{}logout", full_url)).send().await?;
    if logout.status() != 200 {
        println!("Logout unsuccessful, code is {:#?}", logout.status());
    } else {
        println!("Logout successful");
    }

    Ok(())
}

Here's a sample output:

➜  test1 git:(master) ✗ cargo run
    Finished dev [unoptimized + debuginfo] target(s) in 0.09s
     Running `target/debug/test1`
Current firmware image: FL.10.09.1020
Boot firmware image: primary
Logout successful

6. Putting it all together using a dynamic hash-map

use std::collections::HashMap;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let ip_add: &str = "your-ip-address";
    let full_url = format!("https://{}/rest/latest/", ip_add);

    let client = reqwest::Client::builder()
        .cookie_store(true)
        .danger_accept_invalid_certs(true)
        .build()?;
    let params = [("username", "your-username"), ("password", "your-password")];

    let resp = client
        .post(format!("{}login", full_url))
        .form(&params)
        .send()
        .await?;
    if resp.status() != 200 {
        println!("Login unsuccessful, code is {:#?}", resp.status());
    }
     let firmware = client
         .get(format!("{}firmware", full_url))
         .send()
         .await?
         .json::<HashMap<String, String>>()
         .await?;
     println!("{:#?}", firmware);

    let logout = client.post(format!("{}logout", full_url)).send().await?;
    if logout.status() != 200 {
        println!("Logout unsuccessful, code is {:#?}", logout.status());
    } else {
        println!("Logout successful");
    }

    Ok(())
}

Sample output:

➜  test1 git:(master) ✗ cargo run
   Compiling test1 v0.1.0 (/Users/joeneville/code/test1)
    Finished dev [unoptimized + debuginfo] target(s) in 1.56s
     Running `target/debug/test1`
{
    "current_version": "FL.10.09.1020",
    "primary_version": "FL.10.09.1020",
    "secondary_version": "FL.10.09.1010",
    "default_image": "primary",
    "booted_image": "primary",
}
Logout successful

Build an IPv6 Network with Docker Compose

Joe Neville — Fri, 10 Sep 2021 19:04:53 +0000

✍️ In a previous post, I explained how to enable IPv6 for docker and build a network using the command-line
💪 Here I detail how to do the same with docker compose.
👉 Firstly, I build the network via the command-line then docker compose to attach a container to this network.
🔥 Then, I build the network and container service, all in docker compose.
💻 This demo was uses Ubuntu 21.04 server.

Install docker compose

The docker compose installation notes are here. They worked for me without issue.

sudo curl -L "https://github.com/docker/compose/releases/download/1.29.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
sudo chmod +x /usr/local/bin/docker-compose

Example 1: Use an existing network with docker compose

Manually create a network using the command-line. This is a user-defined bridge network. See my previous post for details about this type of network.
```
docker network create --ipv6 \
--subnet="2001:db8:1::/64" \
--gateway="2001:db8:1::1" \
mynetv6-1
```

Create a docker-compose.yaml file that utilises the existing mynetv6-1 network.

version: "3.9"
services:
alp1:
image: alpine:latest
command: ping6 -c 4 2001:db8:1::1
networks:
  - mynetv6-1
networks:
mynetv6-1:
external: true

Notes

This file uses version 3.9.
A service alp1 is created that uses the alpine:latest image. This will ping the network gateway to demonstrate successful network & container build.
There is a networks subcommand under services to assign the service to the mynetv6-1 network.
The dedicated networks section references the existing mynetv6-1, and indicates that this network is already live with the external:true attribute. Without this, docker compose will attempt to create a new network for the alp1 service.

Verify

joe@ub5:~/docker-test$ docker-compose up
Creating docker-test_alp1_1 ... done
Attaching to docker-test_alp1_1
alp1_1  | PING 2001:db8:1::1 (2001:db8:1::1): 56 data bytes
alp1_1  | 64 bytes from 2001:db8:1::1: seq=0 ttl=64 time=0.087 ms
alp1_1  | 64 bytes from 2001:db8:1::1: seq=1 ttl=64 time=0.141 ms
alp1_1  | 64 bytes from 2001:db8:1::1: seq=2 ttl=64 time=0.051 ms
alp1_1  | 64 bytes from 2001:db8:1::1: seq=3 ttl=64 time=0.140 ms
alp1_1  |
alp1_1  | --- 2001:db8:1::1 ping statistics ---
alp1_1  | 4 packets transmitted, 4 packets received, 0% packet loss
alp1_1  | round-trip min/avg/max = 0.051/0.104/0.141 ms
docker-test_alp1_1 exited with code 0

Example 2: Build a network and service with docker compose

This example simply requires a docker-compose.yml file.

version: "2.4"
services:
  alp2:
    image: alpine:latest
    command: ping6 -c 4 2001:db8:a::1
    networks:
      - net2
networks:
  net2:
    name: net2
    enable_ipv6: true
    ipam:
      config:
        - subnet: 2001:db8:a::/64
          gateway: 2001:db8:a::1

Notes

Note the version number, enable_ipv6 is not supported in version 3.
This creates a service, with the latest alpine image, attached to the net2 network, and pings the gateway.
The dedicated network section creates net2, enables IPv6, and addresses the subnet and gateway.

Verify

joe@ub5:~/docker-test$ docker-compose up --remove-orphans
Creating network "net2" with the default driver
Recreating docker-test_alp2_1 ... done
Attaching to docker-test_alp2_1
alp2_1  | PING 2001:db8:a::1 (2001:db8:a::1): 56 data bytes
alp2_1  | 64 bytes from 2001:db8:a::1: seq=0 ttl=64 time=2054.280 ms
alp2_1  | 64 bytes from 2001:db8:a::1: seq=1 ttl=64 time=1053.982 ms
alp2_1  | 64 bytes from 2001:db8:a::1: seq=2 ttl=64 time=53.688 ms
alp2_1  | 64 bytes from 2001:db8:a::1: seq=3 ttl=64 time=0.052 ms
alp2_1  |
alp2_1  | --- 2001:db8:a::1 ping statistics ---
alp2_1  | 4 packets transmitted, 4 packets received, 0% packet loss
alp2_1  | round-trip min/avg/max = 0.052/790.500/2054.280 ms
docker-test_alp2_1 exited with code 0

Final note: Want to keep the container running?

Seeing as this is focused on docker networking, you might want to keep the container up and running, to ping about the network and such like.

Just add command: tail -F anything to your service.
See here for details.

🐦 @joeneville_

Configure Ubuntu WiFi Adapter with Netplan

Joe Neville — Tue, 07 Sep 2021 10:08:45 +0000

👉 Here's a short explanation of how to configure an Ubuntu machine to join a wireless network, with Netplan.
👉 This is for a wireless network with WPA2 Personal authentication (you need a password).
👉 My test machine is running Ubuntu desktop 21.04.

1. Gather Required Details

Get the wireless network details.
This is WPA2 Personal, you're going to need the usual details:
- SSID
- Wireless network password

Get your Ubuntu machine's wireless adapter name.
You can use ip link or ip add for this.

joe@ub1:~$ ip add
...
3: wlx18d6c7116805: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000

In this example, my adapter is wlx18d6c7116805.

2. Compose the Netplan file

Create a netplan yaml file in the /etc/netplan directory.

In this example my file is called mynet1.yaml.
Configure the wireless adapter details under wifis.
The SSID for your wireless network (the name of the network) is configured under access-points.
Record the wireless network password under the SSID.
Make sure to configure any other networking adapters that you require as well, see my previous blog for examples of wired networks here.

Example

network:
  ethernets:
    eno1:
      addresses:
        - 10.150.15.25/24
  wifis:
    wlx18d6c7116805:
      dhcp4: yes
      dhcp6: yes
      access-points:
        "IDontLikeSand15":
          password: "Supersecure123"
  version: 2
  renderer: NetworkManager

Example notes

The wired ethernet adapter eno1 is configured with a static IPv4 address.
The wireless adapter wlx18d6c7116805 is configured for DHCPv4 and DHCPv6 address allocation.
The SSID is IDontLikeSand15 with a password of Supersecure123.

3. Apply the Netplan file

Run the command netplan apply <your-netplan-file-name>.

4. Verify

Use the command iwconfig to check the wireless adapter state.
Use ip add to view all your network adapter.

$ iwconfig
lo        no wireless extensions.

eno1      no wireless extensions.

wlx18d6c7116805  IEEE 802.11bgn  ESSID:"IDontLikeSand15"  Nickname:"<WIFI@REALTEK>"
          Mode:Managed  Frequency:2.412 GHz  Access Point: 24:F2:7F:D1:89:81
          Bit Rate:72.2 Mb/s   Sensitivity:0/0
          Retry:off   RTS thr:off   Fragment thr:off
          Power Management:off
          Link Quality=100/100  Signal level=100/100  Noise level=0/100
          Rx invalid nwid:0  Rx invalid crypt:0  Rx invalid frag:0
          Tx excessive retries:0  Invalid misc:0   Missed beacon:0

$ ip add
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 50:65:f3:2f:c9:a1 brd ff:ff:ff:ff:ff:ff
    altname enp0s25
    inet 10.150.15.25/24 brd 10.150.15.255 scope global noprefixroute eno1
       valid_lft forever preferred_lft forever
    inet6 2001:db8:15:0:5265:f3ff:fe2f:c9a1/64 scope global dynamic mngtmpaddr
       valid_lft 2591913sec preferred_lft 604713sec
    inet6 fe80::5265:f3ff:fe2f:c9a1/64 scope link noprefixroute
       valid_lft forever preferred_lft forever
3: wlx18d6c7116805: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 18:d6:c7:11:68:05 brd ff:ff:ff:ff:ff:ff
    inet 192.168.15.74/24 brd 192.168.15.255 scope link noprefixroute wlx18d6c7116805
       valid_lft forever preferred_lft forever
    inet6 2001:db8:15:0:a402:f49a:d7be:5049/64 scope global temporary dynamic
       valid_lft 599939sec preferred_lft 81131sec
    inet6 2001:db8:15:0:1ad6:c7ff:fe11:6805/64 scope global dynamic mngtmpaddr noprefixroute
       valid_lft 2591914sec preferred_lft 604714sec
    inet6 fe80::1ad6:c7ff:fe11:6805/64 scope link noprefixroute
       valid_lft forever preferred_lft forever

Build a Docker IPv6 Network

Joe Neville — Thu, 26 Aug 2021 19:31:22 +0000

6️⃣ Docker supports IPv6 addressing and IPv6 network builds.
🔇 But IPv6 is not enabled by default.
🔊 Here's how to turn on IPv6.
🧱 Plus how to build three different v6 networks; the default docker0 bridge network, a user-defined bridge network, and an IPvlan network with access to the public Internet.
🎬 TL;DR? Video link at the foot of the page 👇

How to enable Docker IPv6 support

The official docs are here, but I think they could have done a better job with this. Here's my version.

Create a JSON file /etc/docker/daemon.json and write to it a key of ipv6 with a value of true and a key of fixed-cidr-v6 with a value of your chosen IPv6 prefix. This prefix will be assigned to the default docker bridge network, docker 0.
```
{
"ipv6": true,
"fixed-cidr-v6": "2001:db8:abc1::/64"
}
```
Save the file.
Restart docker: systemctl restart docker.

Execute the command docker network ls. Unless you have previously created your own custom networks, you will see the following:

$ docker network ls
NETWORK ID     NAME      DRIVER    SCOPE
9ac6696dded9   bridge    bridge    local
bcc12f998444   host      host      local
32126ee8d073   none      null      local

Inspect the default docker 'bridge' network with docker network inspect bridge. You should see your chosen IPv6 network listed, as well as a gateway address.

joe@ub6:~$ docker network inspect bridge
[
{
    "Name": "bridge",
    "Id": "9ac6696dded9bec310d3451552df8b399503b09b6019b879e08914f6dd056a9d",
    "Created": "2021-08-25T08:10:48.713298521Z",
    "Scope": "local",
    "Driver": "bridge",
    "EnableIPv6": true,
    "IPAM": {
        "Driver": "default",
        "Options": null,
        "Config": [
            {
                "Subnet": "172.17.0.0/16",
                "Gateway": "172.17.0.1"
            },
            {
                "Subnet": "2001:db8:abc1::/64",
                "Gateway": "2001:db8:abc1::1"
            }
        ]
    },
    "Internal": false,
    "Attachable": false,
    "Ingress": false,
    "ConfigFrom": {
        "Network": ""
    },
    "ConfigOnly": false,
    "Containers": {},
    "Options": {
        "com.docker.network.bridge.default_bridge": "true",
        "com.docker.network.bridge.enable_icc": "true",
        "com.docker.network.bridge.enable_ip_masquerade": "true",
        "com.docker.network.bridge.host_binding_ipv4": "0.0.0.0",
        "com.docker.network.bridge.name": "docker0",
        "com.docker.network.driver.mtu": "1500"
    },
    "Labels": {}
}
]

Execute the ip add command and the docker0 interface now shows an IPv6 address:

4: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default
link/ether 02:42:dc:01:46:7a brd ff:ff:ff:ff:ff:ff
inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
   valid_lft forever preferred_lft forever
inet6 2001:db8:abc1::1/64 scope global
   valid_lft forever preferred_lft forever

Create an IPv6 container

Create a container as usual. By default this is assigned to the docker0 bridge network and has an IPv4 & IPv6 address:

$ docker run -di --name alpine6-1 alpine
0f1e634aacf45bfccca1494d6804bbaeac21b870152ceebdaeea8ad72ae27b3d
$ docker exec alpine6-1 ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
   valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
   valid_lft forever preferred_lft forever
22: eth0@if23: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1500 qdisc noqueue state UP
link/ether 02:42:ac:11:00:02 brd ff:ff:ff:ff:ff:ff
inet 172.17.0.2/16 brd 172.17.255.255 scope global eth0
   valid_lft forever preferred_lft forever
inet6 2001:db8:abc1::242:ac11:2/64 scope global flags 02
   valid_lft forever preferred_lft forever
inet6 fe80::42:acff:fe11:2/64 scope link
   valid_lft forever preferred_lft forever

Ping the local gateway:

joe@ub6:~$ docker exec alpine6-1 ping6 2001:db8:abc1::1
PING 2001:db8:abc1::1 (2001:db8:abc1::1): 56 data bytes
64 bytes from 2001:db8:abc1::1: seq=0 ttl=64 time=0.133 ms
64 bytes from 2001:db8:abc1::1: seq=1 ttl=64 time=0.143 ms
64 bytes from 2001:db8:abc1::1: seq=2 ttl=64 time=0.123 ms

The docker0 network is the default bridge network. By default, containers assigned to it will now have IPv4 and IPv6 connectivity to the local gateway and each other.
However, due to limitations with the docker0 network, the advice from docker is that this network should not be used in Production environments, user-defined bridge networks should be used instead. In the next section I will build one such custom network.

User-defined bridge network build

The user-defined bridge network is similar to the docker0 network; it resides on the docker host, and the host acts as the network's gateway.
The containers assigned to a user-defined bridge network have access to other containers on the same network, to the gateway and to external networks.
However, because the docker host performs the role of a gateway, external networks will need to have the necessary routing information to be able to communicate with the custom network. See the diagram below.

In this example, the external server, by default, will not have a route to the user-defined bridge network's subnet, named my-net1 (2001:db8:1::1/64).
The docker host acts as the gateway between the external network (2001:db8:eeee::/64) and my-net1.
The containers attached to my-net1 will be able to send traffic to external networks, using the docker host as their default gateway, but return traffic will fail, the external server will lack the required information to return traffic to the containers.
Therefore, this situation can be improved by adding a static route to the external server, pointing to 2001:db8:1::/64 via the docker host (2001:db8:eeee::/64).

Build Step-by-step

Enable IPv6 for docker, see above for details.

Create a new IPv6 bridge network with the following command:

docker network create --subnet="<your-v6-prefix>" \
--gateway="your-gateway-address" \
--ipv6 \
<name-of-bridge-network>

Attach a container to this network using the --network flag for docker run.
Verify with docker network inspect <your-network>.
For external access, ensure destination networks have a route to <your-v6-prefix>.

Step-by-step example


# Create the network
$ docker network create --subnet="2001:db8:1::/64" \
    --gateway="2001:db8:1::1" \
    --ipv6 \
    my-net1
# Create and attach a container
$ docker run -di --name alpine-1 \
    --network my-net1 \
    alpine:latest
# Verify
$ docker network inspect my-net1
[
    {
        "Name": "my-net1",
        "Id": "c71cbef39206ccaf037db82a12993e59851c1ca9ad18f45975cff8a96f85240a",
        "Created": "2021-09-02T08:13:38.448154937Z",
        "Scope": "local",
        "Driver": "bridge",
        "EnableIPv6": true,
        "IPAM": {
            "Driver": "default",
            "Options": {},
            "Config": [
                {
                    "Subnet": "172.20.0.0/16",
                    "Gateway": "172.20.0.1"
                },
                {
                    "Subnet": "2001:db8:1::/64",
                    "Gateway": "2001:db8:1::1"
                }
            ]
        },
        "Internal": false,
        "Attachable": false,
        "Ingress": false,
        "ConfigFrom": {
            "Network": ""
        },
        "ConfigOnly": false,
        "Containers": {
            "736667de9c88a66fa14aa664eaad827bac7e68776288997361b5cda157949b65": {
                "Name": "alpine-1",
                "EndpointID": "88e56b4c8acbc4342ff0f3c0624a8e801b7fc3f4002339d14450645117ddd5b5",
                "MacAddress": "02:42:ac:14:00:02",
                "IPv4Address": "172.20.0.2/16",
                "IPv6Address": "2001:db8:1::2/64"
            }
        },
        "Options": {},
        "Labels": {}
    }
]

A Note on external communication for IPv6 containers

In this example I've used the IPv6 prefix assigned to documentation for illustrative purposes.
As previously mentioned, for containers on this network to communicate with external destinations, the external nodes will require the routing information to the internal v6 network.
If you wish to provide your containers with access to the public internet, their attached network will need to be a globally routable prefix.
In my home lab, my ISP currently only provides one globally routable prefix, assigned to my LAN. (I have a /56 assigned to my home but the ISP router is currently locked to only act as the gateway for a single /64 on the LAN).
In this case, I opted for an IPvlan Layer 2 docker network, this uses the docker host's IPv6 network. In my case, this network is the ISP assigned globally routable network.

IPvlan network build

The official docs for IPvlan are here

I'm using the default L2 mode.
In this mode, the docker host parent interface will operate at Layer 2 only and switch traffic from containers to the external gateway, in my case this is my ISP router, which will then forward on the traffic to the public internet.

See the diagram below; the parent host interface, the containers, and the external gateway sit in the same IPv6 subnet.

Build Step-by-step

Create the IPvlan network with the following command:
```
docker network create -d ipvlan \
--subnet=<your-network-address>::/64 \
--gateway=<your-gateway-address>  \
--ipv6 -o parent=<your-host-parent-interface> \
<your-network-name>
```
The parent interface sits in the same subnet as the gateway. Check 'ip addr' to confirm the interface name.
The created network is dual-stack, with both IPv4 and IPv6 addressing. Unless you specifically configure the IPv4 addressing, it is a /16 taken from the 172/8 range.
Create a new container and connect it to the IPvlan network.
```
docker run -di --name alpine6-2 \
  --network <your-network-name> alpine
```
This creates a container and attaches it to the IPvlan network.
The container is dual-stack, with an IPv4 address and IPv6 address.
Verify with ip addr.

To test, ping an IPv6 site, such as youtube.com:

$ docker exec alpine6-2 ping youtube.com
PING youtube.com (2a00:1450:4009:815::200e): 56 data bytes
64 bytes from 2a00:1450:4009:815::200e: seq=0 ttl=117 time=11.663 ms
64 bytes from 2a00:1450:4009:815::200e: seq=1 ttl=117 time=16.935 ms
64 bytes from 2a00:1450:4009:815::200e: seq=2 ttl=117 time=11.311 ms

TL;DR? 👇

Still got love for Python format()

Joe Neville — Fri, 12 Feb 2021 12:05:41 +0000

✍️ In my previous post about Python f-strings, I was a little too hard on the older string formatting approach, format().
👀 I was focused solely the string formatting in terms of string substitution, for which I find f-strings the more intuitive and superior approach.
🧰 However, as some devs rightly pointed out, format has some other, very useful, features, and should not be consigned to the mental dustbin. But rather dusted off and placed alongside f-strings in your Python toolkit when it comes to strings.

format() uses

👉 Formatting strings with variable substitution and concatenation

a = "know"
b = "thanks!"
quote = "Yes, we already {} this, {} {}".format(a, b)
print(quote)
Yes, we already know this, thanks!

So far, so f-strings.

👉 String formatting with padding

If you are struggling to structure your string lay-out and presenting them in a way that is pleasing to the eye, format can help. Rather than having to count out 26 whitespaces each time (or was it 25) to get them to line up, you can use padding.
Rather than just use empty curly braces, padding allows you to:

define a field size to be inserted into the string
position your characters
define the fill characters.

In the curly braces, to the right of a colon, add the minimum number of character fields: {:20} this will be 20 fields minimum.

print("Give me some {:20} please!".format("space"))
Give me some space                please!

Use "<" to left-align, ">" to right align and "=" to centre. The default is to left-align, as above:

print("Give me some {:>20} please!".format("space"))
Give me some                space please!
print("Give me some {:^20} please!".format("space"))
Give me some        space         please!

Add in a character to fill the designed number of spaces, the default is just whitespace:

print("Give me some {:*^20} please!".format("space"))
Give me some *******space******** please!

Note that if you want to use your own fill characters, you need to set the alignment as well, not rely on the default. Thus for left-align:

# This will throw an error
print("Give me some {:*20} please!".format("space"))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: Invalid format specifier
# Define the alignment
print("Give me some {:*<20} please!".format("space"))
Give me some space*************** please!

In fact, format uses a "mini-language" that has numerous other options. Check the Python docs for more info.

There's going to be some uses for padding, but if you are trying to align your text into a table, there are Python libraries for that, which are probably a better option, like tabulate.

👉 Accessing Dictionary Values

If you're working with Python dictionaries, format has another useful feature for you. You can access the values of a dictionary directly from format. No need to unpack to variables then reference them, you can call the dictionary inside the format argument.

paint = {"colour": "Sky Blue", "volume": 5}
"For sale, {volume} litres of {colour} paint".format(**paint)
'For sale, 5 litres of Sky Blue paint'

👉 Check this docs for more

format() has even more features that those listed here, check the Python docs for more examples.
Also, I've used this blog from Digital Ocean a few times. It presents information about format() in a style that I found easier to understand than diving straight in the official docs.

In conclusion, f-strings and format(); group hug. 🤗

Use Python f-strings and never look back!

Joe Neville — Mon, 21 Dec 2020 18:28:55 +0000

❓Are you using f-strings to format your strings in Python? Yes? Great.
🚀 If not, consider dropping your tired old format approach and get with the new(ish) stuff: f-strings.
📚 There's plenty of blogs out there that tell you all about f-strings. They tell you at length about the old approaches, % formatting and .format(), so I'm not going to go over all that, I'm just here to encourage you to take a look at f-strings.
🤔 Why? I'm using f-strings all the time now, and .format() seems clunky and overly verbose in comparison.

Here's the basics:

f-strings is the name given to 'formatted string literals'
Python 3.6+
Similar in approach to .format() but less verbose, which helps keep your line length down and improve code readability.

A few examples

Here's a simple combination of string variable declaration and print using .format():

>>> device = "clocks"
>>> time = "thirteen"
>>> sentence = "The {} were striking {}".format(device, time)
>>> print(sentence)
The clocks were striking thirteen

Now, the same code, but using f-string:

>>> device = "clocks"
>>> time = "thirteen"
>>> sentence = f"The {device} were striking {time}"
>>> print(sentence)
The clocks were striking thirteen

Looking at the two approaches it doesn't seem like much of a difference. But I think the improvement are enough to make me switch to f-strings and never look back.

F-strings save you eight characters by using a single 'f' to declare that the code uses f-strings, rather than the '.format()'.
Writing code is more intuitive, I feel, adding the string variables within the string itself, rather than saving this for the end of the line.
Reading code is easier because the declarations are right there in the string, again, rather than at the end of the line.
N.B. .format does allow variables in the braces, but that has the additional requirement of having to declare those in the final brackets, like so:

sentence = "The {device} were striking {time}".format(
device=device, time=time
)

As you can see, this approach is even more verbose.

In isolation, all this points may seem minor, but as string formatting is such a necessary part of so much Python code, these small improvements can make a big difference to the overall experience of reading and writing Python.

Some extra details

Quotation marks can be used within an f-string, as long as different types are used

>>> f'{"quoted string"}'
'quoted string'

Methods can be called directly

>>> sentence = f"The {device.upper()} were striking {time}"
>>> print(sentence)
The CLOCKS were striking thirteen

For more details, check out PEP 498, the proposal that was developed into f-strings here.

Give f-strings a go, I hope you find them as useful as I have.

Thanks for reading.

🐦 @joeneville_

Dnsmasq - Lightweight Name Resolution For Your Home Lab

Joe Neville — Thu, 17 Dec 2020 08:33:55 +0000

If you need name resolution (DNS) for a small network or home lab, Dnsmasq is worth investigating.

💻 Runs on Linux, macOS.
📦 Simple package install with apt-get (on ubuntu linux).
📛 Provides DNS and DHCP services - I'm just running DNS here.

The only downside I've encountered so far is that, while the man page is detailed, some user produced 'how to' guides use various different ways to configure the setup for Dnsmasq.

I've documented the steps that worked for me.

Requirements

I'm building a private Public Key Infrastructure in my home lab.
For the lab I need name resolution.
I'm learning and documenting as I go, all servers and their configurations should to be disposable.
I'd like an approach that allows quick and easy updates to DNS records.

I've found that Dnsmasq is a good fit for my requirements.

Here's my install notes for a simple two server install using Dnsmasq for DNS only. I cover IPv4 & A records only but Dnsmasq supports IPv6:

Dnsmasq Install Step-by-Step

Build two linux server VMs, I'm using Ubuntu 20.04.1.

Update and upgrade

sudo apt-get update && sudo apt-get -y install

Use Netplan (or whichever approach you prefer) to statically configure IP addresses, default gateway, name servers and search domain. See here if you need more information on Netplan.
Note: On all server that are not running Dnsmasq, point their name server configuration to the Dnsmasq server IP address.
Here's an example Netplan YAML file:
```
network:
  ethernets:
    eth0:
      dhcp4: False
      addresses:
        - 192.168.1.21/16
        - "2001:db8:2::21/64"
      gateway4: 192.168.1.254
      gateway6: "2001:db8:2::99"
      nameservers:
        search: [zola.home]
        addresses: [192.168.1.20]
  version: 2
```
On the name server, install Dnsmasq
```
sudo apt-get -y install dnsmasq
```

The package will attempt to start Dnsmasq, this will fail because the port is in use already. This is fine and we will fix later.

Dec 16 10:59:59 name-1 dnsmasq[1165]: dnsmasq: failed to create listening socket for port 53: Address already in use
Dec 16 10:59:59 name-1 dnsmasq[1165]: failed to create listening socket for port 53: Address already in use
Dec 16 10:59:59 name-1 dnsmasq[1165]: FAILED to start up

Now turn off the systemd-resolved and delete the resolved config file.

sudo systemctl stop systemd-resolved
sudo systemctl disable systemd-resolved
sudo rm -v /etc/resolv.conf

Configure Dnsmasq config file /etc/dnsmasq.conf.
Here's a sample config file:

# Never forward plain names (without a domain)
domain-needed
# Turn off DHCP on eth0
no-dhcp-interface=eth0
# Never forward addresses in the non-routable address space (RFC1918)
bogus-priv
# Add domain to host names
expand-hosts
# Domain to be added if expand-hosts is set
domain=zola.home
# Local domain to be served from /etc/hosts file
local=/zola.home/
# Don't read /etc/resolv.conf (I deleted it). Get the external name server from this file, see 'server' below
no-resolv
# External server, works with no-resolv
server=8.8.8.8

Update the Dnsmasq server /etc/hosts file with the name and IP address of the hosts you wish to resolve.
```
127.0.0.1 localhost
192.168.1.20 name-1
192.168.1.21 server-2
```

Start and Enable the Dnsmasq service.

sudo systemctl start dnsmasq
sudo systemctl enable dnsmasq

To test: ping from the Dnsmasq server to an external site, itself and a neighbouring server. Do the same on Server-2.

joe@server-2:~$ ping name-1
PING name-1.zola.home (192.168.1.20) 56(84) bytes of data.
64 bytes from name-1.zola.home (192.168.1.20): icmp_seq=1 ttl=64 time=0.152 ms
64 bytes from name-1.zola.home (192.168.1.20): icmp_seq=2 ttl=64 time=0.221 ms
^C
--- name-1.zola.home ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1023ms
rtt min/avg/max/mdev = 0.152/0.186/0.221/0.034 ms

Configure Ubuntu Networking with Netplan

Joe Neville — Thu, 03 Sep 2020 21:15:36 +0000

Why configure a static IP address

👉 By default, Ubuntu server boots with DHCPv4 enabled.
👉 While dynamic IP addressing is convenient, it is more aligned with clients, that joins and leave networks, rather than servers, which tend to remain fixed on a network.
👉 If you're building anything beyond the most basic deployment, it is wise to plan out your server infrastructure.
👉 That means using static IP addressing for your servers, which requires a little knowledge about how to configure netplan.

Enter netplan

Ubuntu uses netplan to control its networking configuration. It is a powerful tool, consolidating the regular networking configuration points, such as interface addressing and DNS servers, with more advanced features, such as tunneling, interface bonding and security.

Yet more YAML

We configure netplan by writing a YAML file according to a fixed structure, then applying said file.

Go to the netplan examples site to get an idea of the required structure.

Netplan is annoying. Netplan examples? Also annoying!

Netplan itself is pretty annoying. Not only are there no default examples in the base file, the structure and syntax need to be exact. Cue some trial and error to get the yaml to work.
In addition to this, the examples given on the examples page are seemingly a random mix of features. There's a definite bias towards IPv4 on that page as well, IPv6 configuration is not worthy of its own snippet and has to just co-star in some other feature's example.

How to configure netplan

A netplan workflow

As new install of ubuntu server 18.04 & 20.04 will have a default netplan yaml file stored in /etc/netplan.
```
  joe@ubserv2:~$ ls /etc/netplan
  00-installer-config.yaml
```

I usually copy this as a starting point.

sudo cp 00-installer-config.yaml netplan1.yaml

Configure the new yaml file as required. See below for an example.
Netplan can verify the syntax of the new file before applying it with the netplan generate <your-file-name> command.
Once you are ready to go, activate your new networking configuration with netplan apply <your-file-name>
If you are using an SSH connection into the device using the previously configured IP address, your connection will be cut at this point.

To avoid being locked out from your server, use the
netplan try <your-file-name> command. This will apply your selected netplan yaml file but will rollback to the previous configuration if you do not confirm the change within 120 seconds.

A netplan example

Here's an example that will:

Configure static IPv4 and IPv6 on an ethernet interface (eth1).
Configure default gateways for v4 and v6.
Configure v4 and v6 nameservers.
Set a DNS search-list of "mynet.home".

network:
  ethernets:
    eth1:
      addresses:
        - 192.168.20.99/24
        - "2001:db8:20::99/64"
      gateway4: 192.168.20.1
      gateway6: "2001:db8:20::99"
      nameservers:
        search: [mynet.home]
        addresses: [8.8.8.8, "2001:4860:4860::8888"]
  version: 2

Automate Hyper-V VM Creation with Powershell

Joe Neville — Wed, 26 Aug 2020 08:53:14 +0000

If you're using Hyper-V for your virtualization, the Hyper-V Manager GUI provides a serviceable, if a little dull, interface.
However, the bland colour scheme is not the only problem with the graphical interface, its true weakness is just how limited and inefficient it is. Using the wizard, only basic functionality is available, forcing users to click through the presented screens, before going into the virtual machine's 'Settings' to select such rudimentary features as the number of CPUs and Networking configuration.

If you have been paying attention to the messaging coming from Microsoft for years now, it should be no surprise that automating your operations with powershell scripting is a more efficient alternative to all that mouse clicking.

If you dislike powershell, I can sympathize with you, I find it to be quite an ugly customer when compared to the elegance of well-written Python. But, this isn't a beauty contest, we are here to get the job done and leave work on time. Powershell for VM creation it is then!

Sample powershell script for a new VM

Requirements

Specs 📝

A new VM with 2 CPUs, 8GB RAM and 20GB storage.

Networking 🤝

The VM should be attached via its 'port1' to my local vswitch, named "vswitch1".
The port will be an access port in VLAN199.

To Finish ⌛️

Once built, start the VM.

Script

# This script is in two parts. First we declare the variables to be applied.

$vm = "my_new_vm" # name of VM, this just applies in Windows, it isn't applied to the OS guest itself.
$image = "path\to\vm\iso\file.iso"
$vmswitch = "vswitch1" # name of your local vswitch
$port = "port1" # port on the VM
$vlan = 199 # VLAN that VM traffic will be send in
$cpu =  2 # Number of CPUs
$ram = 8GB # RAM of VM. Note this is not a string, not in quotation marks
$path_to_disk = "path\to\disk" # Where you want the VM's virtual disk to reside
$disk_size = 20GB # VM storage, again, not a string

# The following are the powershell commands

# Create a new VM
New-VM  $vm
# Set the CPU and start-up RAM
Set-VM $vm -ProcessorCount $cpu -MemoryStartupBytes $ram 
# Create the new VHDX disk - the path and size.
New-VHD -Path $path_to_disk$vm-disk1.vhdx -SizeBytes $disk_size
# Add the new disk to the VM
Add-VMHardDiskDrive -VMName $vm -Path $path_to_disk$vm-disk1.vhdx
# Assign the OS ISO file to the VM
Set-VMDvdDrive -VMName $vm -Path $image
# Remove the default VM NIC named 'Network Adapter'
Remove-VMNetworkAdapter -VMName $vm 
# Add a new NIC to the VM and set its name
Add-VMNetworkAdapter -VMName $vm -Name $port
# Configure the NIC as access and assign VLAN
Set-VMNetworkAdapterVlan -VMName $vm -VMNetworkAdapterName $port -Access -AccessVlanId $vlan
# Connect the NIC to the vswitch
Connect-VMNetworkAdapter -VMName $vm -Name $port -SwitchName $vmswitch
# Fire it up 🔥
Start-VM $vm

Now the VM will boot and you can complete the install process.

How to run virtual environments with different versions of Python

Joe Neville — Fri, 14 Aug 2020 07:37:03 +0000

Always use a Virtual Environment

🦉 It is a wise move to always run your Python projects in a virtual environment.
🎈 Installing the many dependencies that the average Python program requires into your base OS can soon lead to a bloated system.
🍱 Better to run a virtual environment per project and keep everything clean and compartmentalized.

Python 2 and Python 3?

With that in mind, I had a requirement recently to run a program that required Python 2.7. I was running Python 3.8 on my machine.
I realized that a virtual environment itself wasn't going to be enough. By default my virtual environments were all using Python 3.8.

Here's how I approached the problem on Windows 10:

Install Python2.7 from python.org

Make sure you take a note of where the .exe file is installed. On my Windows machine the path was:
```
C:\Python27\python.exe
```
Install the Python package 'virtualenv' using pip:
```
pip install virtualenv
```
Create a new virtual environment using the Python 2.7 executable.

Here the virtual environment is called venv1 and it uses Python2.7.
```
virtualenv -p C:\Python27\python.exe venv1
```
Activate the virtual environment.
```
.\venv1\Scripts\activate
```
Now if I enter 'python' I can see that I'm using Python 2.7.18 rather than the default 3.8:
```
(venv1) PS C:\Users\joe> python
Python 2.7.18<snip>
```
To create a virtual environment with Python 3.8, I can just run virtualenv without the p flag:
```
virtualenv venv2
```

✨👹✨
@joeneville_

JupyterLab in a virtual environment

Joe Neville — Thu, 09 Jul 2020 16:05:46 +0000

Why?

🏋️‍♀️ When it comes to running JupyterLab & JupyterNotebooks, the install is pretty hefty, with quite a few dependencies.
🦉 To avoid the travails of picking through a messy base install of Python on your dev machine, it is wise to always use a virtual environment.
👍 Thus running JupyterLab in a virtual environment is a good idea.

How?

Here's how using macOS.

Navigate to a suitable directory in which want to work, or make a new one.
Copy the Jupyter Notebook that you want to work on to this directory.
Create a virtual environment and give it a name, in the example below, I'm using 'jup1'.
```
python3 -m venv jup1
```
Activate the the virtual environment.
```
source jup1/bin/activate
```
Now that the venv is active we can install JupyterLab.
```
pip install jupyterlab
```
Start JupyterLab.
```
jupyter lab
```

Extra Tips

The shortcuts for Jupyter Notebooks really help to speed up working with them. It pays to invest some time in learning them. Or, if you can't be bothered with that, just remember that 'shift' + 'return' runs a cell.
If you want to check your notebooks into version control, such as github, make sure you clear the outputs first. To do so right-click on the notebook, or Edit > Clear All Outputs. Then save and do your commits.