DEV Community: Nick Doty

Links to help those impacted by the wildfires in Hawaii

Nick Doty — Mon, 14 Aug 2023 22:42:31 +0000

To locate missing loved ones:1-800-RED-CROSS

Office of the Governor, Hawai‘i Community Foundation

The American Red Cross of Hawaii donation site

Feeding America Food Banks

Maui Food Bank

Help Pets: Maui Humane Society

Building a Discord bot to filter out swear words

Nick Doty — Tue, 29 Sep 2020 16:06:52 +0000

Now that it's been 7 years since quarantine began, I've been expanding my assortment of random Discord bots.
My latest experiment has been a bot that can determine if a message has a "bad word" in it, and if so, deletes the message, and then shames the user. It also can determine if the user is trying to get around the filter by using "leetspeak". This is a short write-up of how that bot works.

Disclaimer: This bot (and the following write-up) revolves around swear words. I will be talking about swear words in this write-up. If that bothers you, I would recommend not reading this. Personally, I have much worse things to worry about in 2020.

Bot information

If you'd like to install the bot on your Discord server, simply click this link. FYI: You must be an administrator on the Discord server to install it. It requires the manage messages permission as it deletes messages with bad words.
If you'd like to take a look at the bot, all of the code is available here on Github.

If you have any requests or bugs to report, you can report an issue here on Github or you can reach out to me on Twitter, @nldoty.

Building the bot

In building this bot there were 4 areas that I focused on in development and I'll be focusing on each shortly below.

Bad word list

The first problem is establishing a list of "bad words". Depending on what you're comfortable with, this can vary quite a bit. You could choose a classic, the 7 words you can't say by George Carlin. Another article I found had 26 swear words listed. But I wanted to be as thorough as possible - so I chose this list of 1300 words from CMU. That being said - there were a lot of words that could be seen as "controversial" that I didn't feel the need to censor. Words like "Republican" and "Democrat". I narrowed it down to about 1000 words.

Data Structure

Next we need a way to see if any words in a message happens to be in the designated list of "bad words".
The naive approach would be a list search. Every time you come across a word, you check to see if it exists in your list of "bad words".
The problem with this approach is time complexity. Searching a python list (x in list) is O(n) where n is the number of items in your list. If your list has only 7 words in it, this might be acceptable! But with over a thousand words, this isn't going to work.
Thankfully, this is almost a textbook use-case for a Trie.

So uhh, what's a Trie?

A Trie, or prefix tree, is an ordered-tree structure used to store dynamic information that can be searched for. It's a unique type of tree because unlike a binary tree, no node in the tree stores the key that you're searching for - instead, the key itself is distributed.

The purpose of this post isn't to teach Tries: so if you'd like to read more, this GeeksforGeeks post is a great resource. But here's a TL;DR:
Tries are made up of nodes, and each node contains two properties: children representing the nodes beneath the node (usually an array), and a way to designate the END of a word. The implementation is stored here, but the code is very simple:

class TrieNode:
    _MAX_SIZE = 40

    def __init__(self):
        self.children = [None] * self._MAX_SIZE
        self.is_end_of_word = False

The _MAX_SIZE is 40 because that is the size of my dictionary:
'abcdefghijklmnopqrstuvwxyz12345670(|@$[!'
The Trie itself has two base functions: insert and search. To "build" the Trie you will insert all of your words into the structure, and then you can search to see if a word is in your Trie.
Here is an example image from TheoryOfProgramming:

So for my example, I want to not only add all my words from my bad words list to my Trie, but I also wanted to add leet speak variations to Trie to catch as many foul words as possible.

To do this, I use some recursion to add extra TrieNodes when I find a letter that has a leetspeak equivalent - passing the remainder of the letters as the word to insert to fill in the Trie with all possibilities.
The code can be seen here.

Let's look at how this would work in practice, with the word shit.
Looking at each individual letter, s h i t:
s has two equivalents, $ and 5
h has no equivalents
i has two equivalents, | and !
t has one equivalent, 7

This ultimately gives us 18 different options to search for:
[shit, shi7, sh|t, sh|7, sh!t, sh!7, $hit, $hi7, $h|t, $h|7, $h!t, $h!7, 5hit, 5hi7, 5h|t, 5h|7, 5h!t, 5h!7]

Searching through 18 different words for one word would be slow. But the Trie is quick and simple, because the complexity is limited to the length of your word.

The more words you're searching for, the more time you'll be saving. This is the beauty of using a Trie.

Discord integration

The Discord integration code can be seen here and is very basic: it listens for all messages to a server, parses the message into words and searches the Trie for each word. If a word is found in the Trie, the bot will delete the message and then shame the user using a random assortment of shaming messages.

The only somewhat interesting part of this code snippet is this line:
text = text.translate(str.maketrans(table))

A major problem I ran into in working on the bot is determining the question, "what is a word?" This was one of the hardest things for me to solve. When people type, they use things like punctuation - commas, periods, dashes, you name it. A typical way to divide a string in python is using .split(), which will break things up by space. However, if you had the sentence
I like spinach, corn, and beans.
Using .split(), your result would be
['I', 'like', 'spinach,', 'corn,', 'and', 'beans.']

In building my Trie, I don't include punctuation like periods and commas. So if you wrote the sentence
"Shut up you stupid fucker."
.split() would break this up as
['Shut', 'up', 'you', 'stupid', 'fucker.']
and my Trie would miss the word fucker because there is a period as part of the word, so I would not see the is_end_of_word at the right point.

To avoid this, I use string.translate. Using the defined dictionary, it maps keys to their respective values - in my case, I map all the common characters that aren't part of my Trie leetspeak to None. This solved quite a few of my problems, but obviously not all of them.

The last major issue I have is one not easily solved. A really easy way to get around the bot is to use spaces in the middle of the word - instead of shit, just write sh it. The bot sees this as two words, and doesn't connect that without the space, a swear word was made. I could add spaces to the Trie at every level, but how would words be ultimately divided? How could I pass words to the Trie so that shit, sh, and it are all checked? This gets more and more complicated the more words/spaces your message has. You could search for all possible iterations with a moving window of sorts, but then you're running into major speed issues.
The best answer would be some way for a program to know that fu ck is really just one word, but oh hm should be treated as two words. I think this problem is best explained using this XKCD comic:

Dockerization

Finally, I like to build my Discord bots into Docker containers. It just makes deployment a lot easier. And the Dockerfile for this one is very simple:


# set base image (host OS)
FROM python:3.7.7

# set the working directory in the container
WORKDIR /code

# copy the dependencies file to the working directory
COPY requirements.txt .

# install dependencies
RUN pip install -r requirements.txt

# copy the content of the local src directory to the working directory
COPY src/ .

# command to run on container start
CMD [ "python", "./main.py" ]

Then, building the container in the top directory is as simple as
docker build -t pottybot .
and running the container is just
docker run pottybot

And that's it!
If you want to run the bot yourself, it's fairly straight-forward. Follow along with this guide to create a bot in the Discord Developer Portal. Once you have a token, create a .env file in the src/ folder with one line:
DISCORD_TOKEN='YOUR_TOKEN_HERE'

Then build and run the Docker container and you're set! If you have any questions or comments, feel free to reach out to me and I'm happy to help!

Stay safe and healthy!

How to secure your Raspberry Pi - or - how my Raspberry Pi was hacked

Nick Doty — Wed, 22 Apr 2020 19:32:36 +0000

If you want to skip past the meat of the article and just read how to better secure your Raspberry Pi, do the following steps:

Don't use the default login of U:pi PW:raspberry.

If you don't need SSH, disable it.

If you do need SSH, consider only using ssh keys, and disabling username/password login via /etc/ssh/sshd_config and changing your PasswordAuthentication field from yes or commented out to

PasswordAuthentication no

If ssh and passwords are a must, take some steps to make brute force attacks harder:

PermitRootLogin no AllowUsers your_username LoginGraceTime 30 MaxAuthTries 1 MaxStartups 2

Really, read the official security guide from raspberrypi.org.

On to my idiocy.

A quick back-story: I host a couple things at home - a personal website, a Jenkins instance and a Minecraft server. I've got a NUC that runs all of these things, and I route traffic through a Raspberry Pi 3 which I use as a reverse proxy and DNS client. Most of my workflow can be accessed through Firefox, so I haven't logged into the NUC or the Pi in quite some time.

On February 18th I received the following email from Verizon Security:

At first, when I saw this, I thought nothing of it. Given my experiences with Verizon in the past, I thought this was related to my self hosting of my website but was nothing serious to worry about. I logged into my Verizon account and was greeted by this poorly formatted security message:

At this point, I was suspicious. The Mirai Malware name was specifically listed, and as someone who has almost 20 IoT devices on their network, I was a bit concerned. However, given all this information, I didn't really know where to start.

I have one of the original Google OnHub routers that I bought years ago. With it comes an app which allows you to do tasks most routers let you do - port forwarding, set static IPs, and do remote restarts.

Another major feature the router supports is viewing devices on your network. You can see IP addresses, give priority speeds to a device, and see the amount of data that has been consumed. Having read up on Mirai and how it operates, I figured it was a decent place to start. Here was a look at my network devices, by consumption, for 7 days between February 11 and February 17:

We use our Apple TV constantly as background noise, so this was kind of shocking. I understand that traffic coming into the network for my website goes through the Raspberry Pi, but 95GB of data is absurd.

I ssh'd into my Raspberry Pi and decided to see what was running, because I could see connections being made in real time through the Google Wifi app. I did this through using the command `top:

Most of this doesn't seem nefarious, but what stuck out was the random curl command (on PID 6658 and PID 7740) and the wget commnad (on PID 7690) all running as root. They would appear and disappear so they were hard to determine, but they definitely weren't traffic to my website. If you hit c while running top it will show the full command being run, and this was the output:

If you go to the IP address in the wget command (please don't) it's an html site with just the word "ghost". If you add the i686 to the URL, it downloads a file. At this point, it's pretty clear that not only do I have major issues, but the issues will continue unless I figure out what I did wrong.

I started searching for how this could happen to my Raspberry Pi. There are two main issues that when combined can make your raspberry pi exploitable right out of the box. The issues being SSH server being enabled and the default username and password for the pi being the same for all Pis. If you have a Raspberry Pi purchased after November 2016 (or have updated it since) then ssh is disabled, and you are required to change the default user and password. But if you just plugged it in, years ago and forgot about it, you could be vulnerable.

By default, the raspberry pi login is:

username: pi password: raspberry

If you have a pi, you should really check if this login still works. I know that I changed the password for this account and disabled it, in favor of an account named nick (duh). However, I definitely have SSH turned on as that's the only way I access my Raspberry Pi. When setting up my reverse proxy, I opened up my Pi to ports 80 and 443 through the Google Wifi app. I decided to check and see if those were still functioning and if I'd changed anything else.

At this point I knew I had made a mistake, because port 22 was open to the world on my Pi. I was playing around with the idea of having a web server for SSH into my NUC, but it never came to fruition. Forgetting to remove port forwarding on 22 to my Pi was mistake number one.

Mistake number two I don't recall, but I checked the users list on my Pi using nano /etc/passwd and found a user named temp. Out of a sheer guess, the password was also temp. I don't recall making this user but having spent many a late frustrating night working on my reverse proxy, there's no doubt in my mind that I did it. The third issue (which I had resolved the week before) was that the url pi.doty.dev routed to my raspberry pi. So anyone could type ssh temp@pi.doty.dev and enter password temp and they'd have access to my entire Raspberry Pi.

Whoops.

According to the Verizon Notice, the Mirai malware can be removed if the device is pulled from the network and power cycled. After multiple attempts I noticed that definitely did not work for me, so I ended up erasing the entire thing using the NOOBS Recovery System and everything now works like a charm.

Now, with a clean-slate Raspberry Pi, I decided I should look into steps I can take so this doesn't happen again (beyond being just a total idiot). Here are some basic steps you can take to protect your Raspberry Pi, or any generic Linux system with an open network:

Prologue: Obviously, don't do all the stuff I did unless you absolutely know what you're doing.
Don't port forward port 22 on your router.
Don't make a user temp with a password of temp on any machine that touches the internet.
Check in on your devices every once in a while to make sure they're not downloading gigabytes of data a day for no reason.

First: Change the default username and password immediately upon using your Pi and connecting it to the internet. It should require that you do this, because pi/raspberry is not a secure combo, especially when everyone knows it.

Next: Make sure your SSH is disabled by default, unless you absolutely need it. If you're using the Pi locally to tinker, you probably don't need SSH turned on.

Next: IF at all possible, use SSH keys for login and ONLY SSH keys. This makes being brute-forced virtually impossible, as only you will have the private key to use. Doing this ensures that you're only likelihood of being compromised is if your host machine is stolen or compromised itself. This article talks about the importance of using SSH keys. If your device needs to be publicly available, you likely cannot disable password login. Nonetheless, you can disable it via editing /etc/ssh/sshd_config and setting the value to no:

PasswordAuthentication no

This field might be yes or commented out, or both. Make sure to uncomment it if so as yes is the default value. If you run into issues this StackExchange post has some useful information.

If you're going to open up SSH to the world, expect brute force attacks. Make sure you're using secure passwords. I'd highly recommend using a password manager for everything. Otherwise, make sure it has some length to it - the longer, the better. Check out this guide.

If SSH is open, you'll need to edit your /etc/ssh/sshd_config. Here are some settings that will help:

PermitRootLogin no AllowUsers your_username LoginGraceTime 30 MaxAuthTries 1 MaxStartups 2

If you want to read more on some smart options to choose when using openSSH, I highly recommend this article and reading the official guide from raspberrypi.org.

Happy coding!