5 Web Discoveries Which Are Pretty Useful

Hi dev.to, in my first article I want to give you a brief summary of 5 tools I recently tried out. If you like this, I will probably release articles of this kind on a monthly basis. If you are keen on more tools in the meantime, you can have a look at Turtle (https://turtle.community), which is where these discoveries come from.

1) High Performance Browser Networking - An Overview about the Basics of the Internet

Ok this is just an online book (https://hpbn.co/), but a pretty useful one, and it is open-source and it is a PWA. This book is on the basic building blocks of the internet, but from a very top-down approach. It is a very good introduction, if you want to get some basic knowledge on TCP, UDP, TLS, HTTP, Websockets, WebRTC and many more. It is a perfect balance between not too technical so that it is fun to read, while also giving you enough vocabulary and basic understanding to dive deeper if you want.

2) Undraw - Open Source Illustrations

This website (https://undraw.co) is incredible. It offers a huge amount of open-source SVG illustrations, which are perfect for creating a landing page. You can easily search for different categories, edit the colors, and download the pictures. They are free of attribution requirements. The nice part is that you can easily edit, combine and compose the elements by opening the SVGs with e.g. Inkscape, so that you can create your own creations. Because these illustrations are SVGs, they have a very small file size and are infinitely zoomable.

3) GoAccess - Open Source Monitoring

This open source tool (https://goaccess.io/) is perfect for visually analyzing your server log files. It reads your logfiles and outputs a html file with a dashboard of graphs showing statistics like, unique visitors, most consumed endpoints, ip addresses, geo-locations, top referrers and many more. The good part of this tool is, that it is incredibly versatile because it does not need an extra server for log collection, you can just tail and pipe your log files to it. It took me only an evening to set up correctly and is much better than ELK stack or Grafana Loki when you do not have a giant infrastructure. You do not need cookies or Google Analytics anymore to get insightful data of your app.

4) Cookie Consent - Open Source Cookie Banner Solution

Cookie Consent (https://github.com/osano/cookieconsent) is an open source solution to display a cookie banner on your website which is compliant with EU or GDPR. You can easily add it to your project as an npm package. It then exposes a very simple API to configure your banner. With a bit of CSS you have your custom cookie banner in no time.

5) Netcup - Very good Hosting Provider

The last discovery in this list is a hosting provider (https://www.netcup.eu/). I have really good experiences with hosting here, because it is reliable, modern and quite cheap. The point is, that when looking for hosting providers you have to go through endless lists and reviews (most of them advertisements) and in most cases you only find the real popular ones. So if you want to self-host check it out, you might like it.

Comments

[Mirosław Farajewicz]

Thanks for GoAccess! I need to check it. I wanted to go into Kibana and Logstash but it's a lot of configuration and as far as they don't lie in the documentation all web servers such as apache or nginx are supported out of the box. Cool.

[thojest]

It took me a few days to discover GoAccess. The point is that setting up ELK or Loki is quite time consuming and they require a special server to store your logdata in some db. While this option is also possible in GoAccess, you also can just pipe your logfiles to it with a simple bash script. When you use Nginx (in my example) it is incredibly easy to set up.

[Mirosław Farajewicz]

do you use some bash script for pushing logs that is available on github? Or could you share it somehow?

[thojest]

Here you go. This is the non-realtime version (Browser does not update dashboard in realtime, but this is also possible. Ask if interested). It will collect all log files, copy to my machine into a single log file

#!/bin/bash
rm html/index.html #remove old files
rm log/nginx.log # remove old files
touch html/index.html # create empty new html

# open ssh connection to host in terminal mode
# and tail all logfiles from the server
#
# now grep through the whole thing and exclude log entries which
# fulfill some pattern defined in excludelist.txt
# (useful for excluding dates, IPs, non-interesting routes,  ...)
#
# 
ssh -t $(whoami)@<HOSTNAME> tail -q -n +0 /var/log/nginx/access* | grep -v -f excludelist.txt > log/nginx.log

#open the created html file (still empty) in browser
google-chrome html/index.html &
# now use goaccess on our collected log file and use some config
# file which describes how to draw the dashboard
goaccess --no-global-config --config-file=./data/goaccess.conf --log-file=log/nginx.log

Press F5 in browser to reload the created html file.
There is much room for optimization :) but it just works and I am happy with it :)