DEV Community: Martin André

Block most ads on any device (Wireguard + Pi-Hole)

Martin André — Thu, 13 May 2021 11:56:37 +0000

We all know how ads can be annoying and blocking them typically involve installing an ad-blocker on each device, browser, ... In this guide I'll show you how you can block ads on most device (iPhone, Mac, Android, Windows, ...).

Overview

Pi-Hole is a general purpose network-wide ad-blocker that protect your network from ads & trackers. It's main advantage over browser's ad-blocker is that it block ads on any type of software.

Prerequisites

A VPS (near you or in the country you want to be)
Know the basics of Linux
Some 30 free minutes in your schedule

There is plenty of VPS provider, I've personally chosen Hetzner but Linode is also really good ! (you can use my Hetzner link to get $20 cloud credits)

Setting up Wireguard

It takes no time to install Wireguard on Linux thanks to angristan.

curl -O https://raw.githubusercontent.com/angristan/wireguard-install/master/wireguard-install.sh
chmod +x wireguard-install.sh
./wireguard-install.sh

Go read the README to learn how to use the script.

Setting up Pi-Hole

All you need is running this simple command:

curl -sSL https://install.pi-hole.net | bash

During the setup you will be able to choose the interface for Pi-Hole to listen to, choose wg0.

Optional: install Unbound

sudo apt install unbound

You might have to configure Unbound for it to be fasttttt.

vim /etc/unbound/unbound.conf.d/pi-hole.conf

server:
    verbosity: 0
    interface: 127.0.0.1
    port: 5335
    do-ip4: yes
    do-udp: yes
    do-tcp: yes
    do-ip6: yes
    prefer-ip6: no
    harden-glue: yes
    harden-dnssec-stripped: yes
    use-caps-for-id: no
    edns-buffer-size: 1472
    prefetch: yes
    prefetch-key: yes
    minimal-responses: yes
    cache-min-ttl: 300
    cache-max-ttl: 86400
    serve-expired: yes
    msg-cache-size: 50m
    rrset-cache-size: 100m
    num-threads: 1
    so-reuseport: yes
    so-rcvbuf: 4m
    so-sndbuf: 4m
    private-address: 192.168.0.0/16
    private-address: 169.254.0.0/16
    private-address: 172.16.0.0/12
    private-address: 10.0.0.0/8
    private-address: fd00::/8
    private-address: fe80::/10

You can finally restart unbound.

sudo service unbound restart

Optional: Configure Pi-Hole for Unbound

As you use Unbound, you will have to disable Pi-Hole DNS cache as well as redirecting to the right DNS server instead of using Cloudflare, ...

vim /etc/pihole/setupVars.conf

WEBPASSWORD=
BLOCKING_ENABLED=true
ADMIN_EMAIL=
WEBUIBOXEDLAYOUT=traditional
WEBTHEME=default-dark
PIHOLE_INTERFACE=wg0
IPV4_ADDRESS=
IPV6_ADDRESS=
QUERY_LOGGING=true
INSTALL_WEB_SERVER=true
INSTALL_WEB_INTERFACE=true
LIGHTTPD_ENABLED=true
CACHE_SIZE=0
DNSMASQ_LISTENING=single
PIHOLE_DNS_1=127.0.0.1#5335
DNS_FQDN_REQUIRED=true
DNS_BOGUS_PRIV=true
DNSSEC=false
REV_SERVER=false

And finally you can repair Pi-Hole using:

pihole -r

You can now go on your Pi-Hole dashboard: http://ip/admin.
And check that your settings are correctly configured.

Dashboard > Settings > System > FTL Information
>>> DNS cache size should be = 0.

Dashboard > Settings > DNS > Upstream DNS Servers
>>> Custom 1 (IPv4) = 127.0.0.1#5335
>>> Everything else should be unchecked.

Dashboard > Settings > DNS > Interface listening behavior
>>> Listen only on interface wg0.

Conclusion

And that's pretty much it!

All you have to do now is to generate a config client for your Wireguard server, install it on any device and once the connection will be established, you can say bye bye to ads and hello to anonymity.

Set up Wireguard / Pi-Hole and OpenVPN together

Martin André — Fri, 04 Dec 2020 10:45:47 +0000

Wireguard is fairly new but already ready to replace OpenVPN. It provides a secure connection tunnel from a client to a server using public and private key authentication.

In this tutorial I will assume that you already have some basic knowledge of networking and command line.

What will we do today?

The other day I was struggling configuring my Wireguard instance to use Pi-Hole while also using the Cloudflare DNS and my company's DNS over a OpenVPN connection to reach the servers of the company.

As a result I decided to write my guide, based on my experience. The little graph below resume what we'll end up with.

Setting up Pi-Hole

As the name of the project tends to pretend, Pi-Hole is not only reserved for Raspberry Pi. You can run it on a traditional server too and that's what we're going to do.

All you need is running this simple command:

curl -sSL https://install.pi-hole.net | bash

If you need more information for the install, check out this guide.

Once it's installed, head towards the web admin page of the Pi-Hole and go on the Settings page > DNS.

Here you can select which Upstream DNS servers you want to use and setup your own DNS too.
So in my case, my company DNS address is 10.51.1.1which result in this configuration:

This config allows me to use 1.1.1.1 for general requests and 10.51.1.1 when it's linked to my company (by the domain name).

You're done with Pi-Hole for the DNS, you might want to play with it a bit to block ads correctly.

Open-VPN

In our network graph the Open-VPN connection is only used to speak with my company network. It's running Open-VPN due to our router running pfSense.

To set it up it's pretty easy, you just have to get your config.ovpn file.

Then install openvpn:

apt-get install openvpn

Define your credentials:

echo "username" >> /etc/openvpn/credentials
echo "password" >> /etc/openvpn/credentials

And start the tunnel:

openvpn --config /path/config.ovpn --daemon

You should see a new tun0 interface when you type:

ip a

Wireguard server

Let's install and configure out Wireguard instance now!

The install process is just 3 commands long:

apt-get install linux-headers-$(uname --kernel-release)
add-apt-repository ppa:wireguard/wireguard
apt-get update && apt-get install wireguard

Now we can configure Wireguard.
Start by creating the needed folder and the private/public keys of the server:

mkdir -p /etc/wireguard/keys
cd /etc/wireguard/keys
umask 077
wg genkey | tee privatekey | wg pubkey > publickey

We'll now create /etc/wireguard/wg0.conf which is our Wireguard config:

vim /etc/wireguard/wg0.conf

PrivateKey = private_key # from the step above
Address = 172.16.0.0/12,fd5b:5840:9e9f:a477::1/64 # you can change it, but IT STAY PRIVATE IPS
ListenPort = 8999
PostUp = sysctl -w net.ipv4.ip_forward=1; iptables -A FORWARD -i %i -j ACCEPT; iptables -t nat -A POSTROUTING -o ens2 -j MASQUERADE; iptables -t nat -A POSTROUTING -o tun0 -j MASQUERADE; ip6tables -A FORWARD -i %i -j ACCEPT; ip6tables -A FORWARD -o %i -j ACCEPT; ip6tables -t nat -A POSTROUTING -o he-ipv6 -j MASQUERADE
PostDown = iptables -D FORWARD -i %i -j ACCEPT; iptables -t nat -D POSTROUTING -o ens2 -j MASQUERADE; iptables -t nat -D POSTROUTING -o tun0 -j MASQUERADE; ip6tables -D FORWARD -i %i -j ACCEPT; ip6tables -D FORWARD -o %i -j ACCEPT; ip6tables -t nat -D POSTROUTING -o he-ipv6 -j MASQUERADE

[Peer]
PublicKey = public_key_client_one
AllowedIPs = 172.16.66.2,fd5b:5840:9e9f:a477::ca:571e/128 # update if you changed the Address from above

[Peer]
PublicKey = public_key_client_two
AllowedIPs = 172.16.66.3,fd5b:5840:9e9f:a477::746f:786f/128 # update if you changed the Address from above

Once it's configured, make it start and launch at boots:

 systemctl enable wg-quick@wg0.service
 systemctl start wg-quick@wg0.service

Wireguard client

As it's not the main goal of this tutorial and as it's not very complicated, I'll just give you an example of a client's config.

[Interface]
PrivateKey = client_private_key
Address = 172.16.66.3/32,fd5b:5840:9e9f:a477::746f:786f/64
DNS = 10.18.1.57 # IMPORTANT (IP OF THE PI-HOLE)

[Peer]
PublicKey = server_public_key
AllowedIPs = 0.0.0.0/0,::/0 # ROUTE ALL TRAFIC
Endpoint = 123.123.123.123:8999 # IP OF THE SERVER:PORT
PersistentKeepalive = 15

Rust vs Go - Load testing webserv (>400k req/s)

Martin André — Mon, 16 Nov 2020 23:12:33 +0000

TLDR: Go can reach 270k req/s where Rust can hit 400k req/s.

Go server

Let's go straight to buisness with a minimal server sample using httprouter.

package main

import (
    "fmt"
    "net/http"

    "github.com/julienschmidt/httprouter"
)

func Index(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
    fmt.Fprint(w, "Welcome!")
}

func main() {
    router := httprouter.New()
    router.GET("/", Index)

    http.ListenAndServe(":8080", router)
}

Rust

For the Rust server we'll use Actix as our framework of choice, down below is the minimal sample.

use actix_web::{web, Responder, middleware, App, HttpServer};

async fn health_check() -> impl Responder {
    "Welcome!"
}

fn routes(cfg: &mut web::ServiceConfig) {
    cfg.route("/health", web::get().to(health_check));
}

#[actix_web::main]
async fn main() -> std::io::Result<()> {
    let serv = HttpServer::new(move || {
        App::new()
            .wrap(middleware::Compress::default())
            .configure(routes)
    });
    serv.bind("127.0.0.1:8080")?
        .run()
        .await
}

Benchmark

To put a big load on both our servers, we're going to use wrk.

the benchmarks are performed on a i7-8750H (6c, 12threads)

wrk -t12 -c1000 -d15s http://127.0.0.1:8080/

Results

Rust:

Running 15s test @ http://127.0.0.1:8080/
  12 threads and 1000 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     3.73ms    4.70ms  57.76ms   85.86%
    Req/Sec    33.66k     5.80k   69.35k    71.65%
  6039978 requests in 15.10s, 714.26MB read
Requests/sec: 400095.92
Transfer/sec:     47.31MB

Go:

Running 15s test @ http://127.0.0.1:8080/
  12 threads and 1000 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     5.03ms    6.11ms 102.78ms   86.66%
    Req/Sec    22.81k     4.77k   53.73k    71.19%
  4087276 requests in 15.10s, 487.24MB read
Requests/sec: 270691.36
Transfer/sec:     32.27MB

The results speak for themselves... 400.000 vs 270.000 for Rust and Go respectively.

Conclusion

While Go might be easier to write and faster to compile compared to Rust, it's still slower compared to its competitors.

If you're hesitating, let me give you this advice: use rust if you want speed, else go with Go.

Cover image from dzone.

Get simple IO stats using Rust (throughput, ...)

Martin André — Wed, 11 Nov 2020 17:06:29 +0000

The other day I wanted to grab some information about my disk performance in real-time. I saw the really useful command iostat and got curious. How does this tool can know how many megabytes per seconds my disks is using, etc.

So I dug into the source code of this command and found my answer.

Basically it just read /proc/diskstats and that's all (if we don't care about formatting, sorting, etc).

As I'm learning Rust (and doing a project involving stats) I thought it would be great to write a simple program to only display how many megabytes per seconds my disks are using. And so my journey began.

Setting up the project

We can create our project with cargo. Cargo is a package manager for Rust and you can use it to simplify your life.

cargo new nameofbinary --bin
cd nameofbinary

(if you've never used Rust before, take a look at @aligoren 's fantastic guide here)

Let's code our program

What will we need ? First take a look at the /proc/diskstats;

 259       0 nvme0n1 93188 31843 6970296 27235 1834264 427303 105983322 432380 0 411608 472996 0 0 0 0 33212 13380

Wow. What a mess. However we can easily understand it by a little bit of googling (doc from kernel.org).
By reading it we understand that each value is organized and separated by whitespace. Great !

Now we can start coding (we'll do everything in the main.rs).
We'll first create a struct which will handle the number of megabytes one disk has read or written.

// This structure will hold our data for the disks
struct IoStats {
    pub mb_read: f64, // f64 cause it won't be an integer, we need precision
    pub mb_wrtn: f64,
}

Here is the pseudo code of our program first;

hold_prev = previous stats reference
file = file descriptor to /proc/diskstats
loop:
    hold_curr = current stats reference
    io_stats = content of the file
    for line in io_stats:
        split_field = line.split (get each value in an array)
        compute_mb_s() = calculate the diff between hold_prev and hold_curr
    write_result
    hold_prev = hold_curr
    wait 1s

Let's dive into the real code now;

// Hashmap of previous drives stats to compute difference from
let mut prev: HashMap<String, IoStats> = HashMap::new();
// Open the file we'll use to get the stats
let mut fd = File::open(&"/proc/diskstats").unwrap();

prev is a hashmap which will help us find the prev disk more easily by checking only by the disk name.
fd is our file descriptor (our way to read the diskstats file).

// Main program loop
loop {
    // Create the curr Hashmap, allow us to compare with the prev one
    let mut curr: HashMap<String, IoStats> = HashMap::new();

    ...

    prev = curr;
    thread::sleep(Duration::from_secs(1));
}

curr is the same hashmap type as prev and will hold current value of disks stats.
prev = curr is where we give an utility to prev (now prev contains all the value as curr)
thread::sleep pause the loop for 1s (this avoid spamming the cpu/terminal)

let mut curr: HashMap<String, IoStats> = HashMap::new();
// Create the output string
let mut output = String::new();
// Add the header string to the output
output.push_str("\nDevice          mb_reads/s      mb_wrtn/s\n\n");
// Create a new empty string
let mut io_data = String::new();
// Read the content of the file (diskstats) to the io_data string
fd.read_to_string(&mut io_data).unwrap();
// Iterate over each line (each disk)
for line in io_data.lines() {

    ...

}
// Move the cursor to the start of the file
fd.seek(SeekFrom::Start(0)).unwrap();
// Print the result
writeln!(io::stdout().lock(), "{}", output);
prev = curr;

output is the string we'll print (single string to print everything at once)
output.push_str add the header to the output string
fd.read_to_string read the content of fd into io_data
for iterate over each line from io_data
fd.seek reset the cursor position to the start of the file (allow use to read it again)
writeln! write the output string in stdout (terminal)

for line in io_data.lines() {
    // Split field (separated by whitespace) and collect them without specific type
    let fields = line.split_whitespace().collect::<Vec<_>>();
    let ds = IoStats {
        mb_read: fields[5].parse::<f64>().unwrap() / 2048.0,
        mb_wrtn: fields[9].parse::<f64>().unwrap() / 2048.0,
    };
    // If prev already contains the info we compute the diff to get mb/s
    // Else we add to the print line the "fake" data.
    if prev.contains_key(fields[2]) {
        // Get the object from the hashmap
        let pds = prev.get(fields[2]).unwrap();
        // Construct speed line and append it to curr hashmap
        let mb_read_s = ds.mb_read - pds.mb_read;
        let mb_wrtn_s = ds.mb_wrtn - pds.mb_wrtn;
        // Add the line, formatted with color and spacing
        output.push_str(&format!("\x1b[0;32m{:16}\x1b[0m\x1b[0;34m{:10.2}{:15.2}\x1b[0m\n", fields[2], mb_read_s, mb_wrtn_s));
        // Insert the current disk data to the curr HashMap
        // the curr will later be saved as prev
        curr.insert(fields[2].to_owned(), ds);
    } else {
        // Add the line with fake data and formatting
        output.push_str(&format!("\x1b[0;32m{:16}\x1b[0m\x1b[0;34m{:10.2}{:15.2}\x1b[0m\n", fields[2], 0.00, 0.00));
    }
}

line.split_whitespace().collect::<Vec<_>>() simply return us each values, without whitespace, as &str
ds create our IoStats struct for this disk
ds -> mb_read/mb_wrtn are first converted to f64 and then divided by 2048.0 to convert from sector (one sector is almost always 512kb) to mb.
prev.contains_key check if the previous hashmap contains the disk (in case of first run or usb stick)
pds is the IoStats of the previous run of the loop for that particular disk
mb_read_s and mb_wrtn_s are the mb/s value (/s as we loop every 1s)
output.push_str add the current disk's info to the output string
curr.insert add the current disk's info to the curr hashmap

Final words

As we saw it's pretty easy to get stats from our disks in Rust. Everything is properly placed inside /proc/diskstats for us.

If you want to take a look at the full code github.

Thanks for ready it, hope you learned something new ! ❤