DEV Community: BiasedKiwi

Working with environment variables in Python.

BiasedKiwi — Sat, 30 Apr 2022 09:54:42 +0000

Often times when developing software, we find the need to read values defined by other pieces of software, or even define these variables ourselves! This article will talk all about that.

Accessing global environment variables

Using `os.environ`

We can get and set environment variables using the os.environ dictionnary. Getting the value of an environment variable:

>>> import os
>>> os.environ["HOME"]
'/home/username'

Setting the value of an environment variable:

>>> import os
>>> os.environ["FOO"] = "bar"
>>> os.environ["FOO"]
'bar'

Because os.environ is a mapping object, we can use .pop() to delete a variable from os.environ:

>>> import os
>>> os.environ["FOO"] = "bar"
>>> print(os.environ.get("FOO"))
bar
>>> os.environ.pop("FOO")
'bar'
>>> print(os.environ.get("FOO"))
None

If the environment variable doesn't exist, Python will raise a KeyError, to mitigate this, we can use os.environ.get() which returns None if the environment variable doesn't exist:

>>> import os
>>> bar = os.environ.get("BAR")
>>> print(bar)
None

A neat trick of os.environ.get() is setting default values. This value will be returned if the variable doesn't exist. We can achieve this by passing the default value as the second parameter of os.environ.get()

>>> import os
>>> bar = os.environ.get("BAR")
>>> print(bar)
None
>>> bar = os.environ.get("BAR", "bar")
>>> print(bar)
bar

Using `os.getenv()`

Now you might be wondering: "What is the difference between os.environ.get() and os.getenv()?", well the answer is simply that there is no difference. In terms of performance, both functions perform relatively similarly:

$ python3 -m timeit -s "import os" "os.environ.get('HOME')"
500000 loops, best of 5: 730 nsec per loop
$  python3 -m timeit -s "import os" "os.getenv('HOME')"
500000 loops, best of 5: 809 nsec per loop

os.getenv's usage is quite similar to os.environ.get(), it has the same parameters and behavior:

>>> import os
>>> bar = os.getenv("BAR")
>>> print(bar)
None
>>> bar = os.getenv("BAR", "bar")
>>> print(bar)
bar

Accessing variables in a `.env` file

Sometimes, we would like to write our own environment variables without interfering with other programs, whether it's to store sensitive data or just configuration options, a .env file is what you need.

Accessing the file using `configparser`

The main drawback of using configparser with .env files is that configparser requires sections. Which means that your .env file will have to look like this:

[SECTION1]
VAR_1=foo

Accessing the variables:

>>> import configparser
>>> config = configparser.ConfigParser()
>>> config.read(".env")
>>> var_1 = config["SECTION1"]["VAR_1"]
>>> print(var_1)
"foo"

Accessing the file using `python-dotenv`

python-dotenv is a Python library which allows us to read environment variables from a .env file using the os module:

# Our .env file
FOO=bar

>>> import os
>>> from dotenv import load_dotenv
>>>
>>> load_dotenv()
>>> foo = os.getenv("FOO")
>>> print(foo)
"bar"

load_dotenv() will search for a .env file in the current directory or parent directories by default, and then loads them to make them accessible by your project. But you can also manually specify a directory:

...
load_dotenv("path/to/.env")
...

Use case: Managing sensitive data

Let's say you were working with API tokens, and didn't want to hardcode them into your code, and risk accidentally pushing it to some version control software. You would use environment variables to store your token.

# .env
TOKEN=abcdefg12345

import os
from dotenv import load_dotenv

load_dotenv()
token = os.getenv("TOKEN")

# Do stuff with the token
...

Conclusion

In this tutorial we learned how to use the os module to access host environment variables, and also how to access variables stored in a .env file using configparser and python-dotenv.

Chapter 3. Cogs

BiasedKiwi — Mon, 25 Apr 2022 15:13:16 +0000

Now that we have a general idea of how Discord bots are structured, we can take a look at cogs. But first:

What are cogs?

Cogs are "extensions" for your bot, they behave like Python modules. They can contain commands, and event listeners. Let's take a look at an example.

The anatomy of a Cog.

If you are following along with this tutorial, you will have a file named miscellaneous.py located in src/cogs. When we open it, we find a class declaration of a class named Miscellaneous. Each cog contains 2 parts: the cog class, and the setup function. Let's take a look at the class first.

class Miscellaneous(commands.Cog):
    """Cog class containing all miscellaneous commands."""

    def __init__(self, bot: commands.Bot):
        self.bot = bot

Here, we can see Miscellaneous inherits properties from commands.Cog, a class which all cogs should inherit from. Further down, we can see a function declaration for cog_load

async def cog_load(self):
        """Method called when the cog has been fully loaded."""
        print("The Miscellaneous cog is loaded.")

As it's name suggests, this function is a special function that is called whenever that specific cog has been fully loaded. But the good stuff has yet to come, here, we see our first command. Wondering about slash commands? We'll talk about that later.

@commands.command(
        name="ping", description="Get the current latency of the bot."
    )  # This could be implemented as a slash command.
    async def ping(self, ctx: commands.Context):
        """Get the current latency of the bot."""
        await ctx.channel.send(
            f"The current latency of the bot is {round(self.bot.latency * 1000, 1)}ms."
        )

Woah! This code can be confusing at first (Why is there an @ in the code???), but I'll walk you through it, first we see a function decorator, if you don't know what that is, it's fine. We can also see that the name and description parameters are being passed to commands.command. Then we see the actual function declaration of ping(), which takes ctx as an argument, as the type hint suggests, this argument is of type commands.Context. According to the docs:

Represents the context in which a command is being invoked under.

Then, we see a statement which calls the function send() from ctx.channel. We are also passing a argument, a an f-string to which we give the value of self.bot.latency.

That was alot to take in huh? So what does this command actually do? It sends the bot's latency. Yep.

If you didn't quite catch how commands worked (I mean, they are kinda the whole point of Discord bots), don't worry, the next article in the series is going to be entirely about them.

The setup function

At the end of miscellaneous.py, we can find a function declaration for setup()

async def setup(bot: commands.Bot):
    await bot.add_cog(Miscellaneous(bot))

This function is mandatory for any cog to be loaded, if it is missing, you'll be greeted with a nice error message:

discord.ext.commands.errors.NoEntryPointError: Extension 'src.cogs.miscellaneous' has no 'setup' function.

A setup function has to be an async function. (Note: this is only as of Discord.py v2.0, which we have installed earlier. If you are using an older version, do not make the setup function async). Inside the setup function, we find an async call to commands.Bot.add_cog(cog) where the cog parameter is the cog class.

Loading cogs

Cogs on their own aren't really useful if they aren't loaded. We can load them using commands.Bot.load_extension(name) where name is the name of the cog we want to load. Since cogs behave the same way as python modules, we need to refer to them as such: When you import a module, you don't do it like this:

import path/to/module

But like this:

import path.to.module

So when you use commands.Bot.load_extension(), you have to use the same notation (it's called dot notation) as when you're trying to import a module.

Conclusion

In this article we learned about cogs. At this point in the series, you might be wondering: "When the heck are we going to write commands?", well rest assured, it's coming in the next article. Stay tuned!

Chapter 2. A closer look at the code.

BiasedKiwi — Mon, 25 Apr 2022 10:03:45 +0000

Now that we have prepared our virtual environment, we are ready to start developing our Discord bot.

First, clone a template I have made with git. In the same directory as our virtual environment, run the following commands:

git init .
git remote add origin https://github.com/BiasedKiwi/discordpy-template.git
git pull origin main

Let's take a look at what these commands are doing.

git init . creates a Git repository in the current directory.
git remote add origin sets https://github.com/BiasedKiwi/discordpy-template.git as the remote server.
git pull origin main fetches all the commits on the main branch

Now, these command don't show any output to the console, to check that the remote has been properly registered, let's execute git remote -v. You should see this output:

origin  https://github.com/BiasedKiwi/discordpy-template.git (fetch)
origin  https://github.com/BiasedKiwi/discordpy-template.git (push)

After you've executed all of these command, your directory structure should look like this:

├── LICENSE
├── README.md
├── launcher.py
├── requirements.txt
└── src
    ├── __init__.py
    ├── bot.py
    └── cogs
        └── miscellaneous.py

A look at the code.

Let's take a look at src/bot.py. First, we can see a class declaration:

class MinimalBot(commands.Bot):
    """Constructor class for the bot."""

    def __init__(self, *args, **kwargs):
        self.prefix = kwargs["command_prefix"]
        super().__init__(
            *args, **kwargs
        )  # Initialise a `Bot` instance with the parameters passed in the constructor.

This code declares a class named MinimalBot which inherits properties from discord.ext.commands.Bot. In MinimalBot's __init__ function, it creates an instance of it's parent class, a.k.a. commands.Bot, with the parameters suupplied (*args, **kwargs).

If we continue reading, we can see a function definition:

async def setup_hook(self):
        await self.load_cogs(client=self, directory="./src/cogs")

setup_hook() is a special function called when the bot is being set up. It can be used to perform async operations when the bot first starts, without having to mess with __init__().Inside we can see a call to another function: self.load_cogs()

async def load_cogs(self, client, directory: str, subdir: str = "") -> None:
        """Load all cogs in a given directory in a recursive fashion."""
        os.chdir(directory)
        base = os.getcwd()
        for file in os.listdir():  # Iterate through all the files in a directory
            if file.endswith(".py"):
                if subdir != "":
                    await client.load_extension(
                        f"src.cogs.{subdir}.{file[:-3]}"
                    )  # As of Discord.py 2.0, `load_extension` is a coroutine.
                else:
                    await client.load_extension(
                        f"src.cogs.{file[:-3]}"
                    )  # Refer to comment on line 61.
            elif os.path.isdir(os.path.join(base, file)):
                os.chdir(os.path.join(base, file))
                await self.load_cogs(client, os.getcwd(), subdir=file)  # Recursive call
                os.chdir(base)

Unlike setup_hook(), self.load_cogs() isn't a special function of commands.Bot, it has been defined by us. This function loads all the cogs in a directory. For now, just know that cogs are "extensions" of the bot that can contain commands and other interesting things.

Further down, we can see another function definition: on_ready(). on_ready() is an event that is called when the bot has connected to Discord servers and is done preparing the data sent by Discord.

Events are special functions called when a specific event has been dispatched from Discord (duh!), in this example, on_ready is called when the bot is ready to send and receive data to Discord. Other events also exist like on_message() which is an event that is called whenever a message has been sent in a guild in which the bot is in. For a list of all events, take a look at the link to the Discord.py event reference in the Further Reading section at the end of this article.

Now, let's take a look at another file: launcher.py. This file is very simple, it contains a single main function.

def main():
    intents = discord.Intents.default()
    intents.message_content = True
    bot = MinimalBot(
        intents=intents,
        command_prefix=os.environ.get("BOT_CMD_PREFIX"),
    )
    bot.run(os.environ.get("BOT_DISCORD_TOKEN"))

The first thing that this command does is set the intents, these are very important in Discord bot development, and we will talk about them in the next section of this article. Then, it creates an instance of MinimalBot inside a variable named bot. Finally, it runs the bot using an environment variable to get the token of the bot. (More on that later.)

Intents

Intents were implemented by Discord to regulate the amount of data your bot has access to, for example, we've mentionned the on_message event before. To get this event working, we have to set the message_content intent to True. This will allow your bot to read the contents of all messages the bot can read. But there's a catch, the message_content intent is what Discord calls a privileged intent. This means that once your bot reaches 75 guilds, Discord will have to verify your bot for you to continue using the intent.

Tokens

Tokens are a string of characters that authentify you with the Discord API (Kind of like a password). They carry your autorisations, like privileged intents. Tokens are highly sensitive pieces of info, if it falls in the wrong hands, it could have desastrous consequences for your bot, so make sure to keep it somewhere safe, and only share it with people you truly trust.

Conclusion

Alright! I think this article has come to an end, in the next one, we will look at cogs: how to use them, load them and their gotchas. See you on the next one!

Chapter 1. Initial Setup

BiasedKiwi — Mon, 25 Apr 2022 08:31:01 +0000

Have you ever been on Discord? If yes, then you've probably seen a Discord bot before, they appear as regular members in a server, except with a "BOT" tag next to their username. In Discord, a bot can be summoned by sending a command in chat.

Prerequisites

This tutorial expects you to have at least a decent understanding of Python. Rest assured, even if we will use async syntax, an understanding of asynchronous programming is not needed.

Preparing our dev environment.

First things first, we'll need to install a version of Python higher than 3.8. If you don't already have it installed, you can install it here.

Once you've installed Python, you can check if your python executable is working properly by making it print the Python version. On linux/macOS:

$ python3 --version

Windows:

C:\> python --version

If your Python executable is working properly, you should see this:

Python 3.8.10  # This number can be higher, but not lower.

This step is optional, but highly recommended. We will create a
virtual environment using venv. In your Discord bot's root directory, execute this command:

python3 -m venv .venv  # Linux/macOS

Windows:

C:\> python -m venv.venv

We can activate our environment using source on Linux/macOS

$ source /.venv/bin/activate

On Windows:

C:\> /.venv/Scripts/activate.bat

Almost there! All we have to do now is to install Discord.py v2.0 using pip. Linux/macOS:

$ python3 -m pip3 install -U git+https://github.com/Rapptz/discord.py

Windows:

C:\> python -m pip install -U git+https://github.com/Rapptz/discord.py

Once that's done, you did it! We have successfully prepared our environment to start writing our Discord bot, see you in the next tutorial!

DEV Community: BiasedKiwi

Working with environment variables in Python.

Accessing global environment variables

Using os.environ

Using os.getenv()

Accessing variables in a .env file

Accessing the file using configparser

Accessing the file using python-dotenv

Use case: Managing sensitive data

Conclusion

Further Reading

Chapter 3. Cogs

What are cogs?

The anatomy of a Cog.

The setup function

Loading cogs

Conclusion

Further Reading

Chapter 2. A closer look at the code.

A look at the code.

Intents

Tokens

Conclusion

Further Reading

Chapter 1. Initial Setup

Prerequisites

Preparing our dev environment.

Further Reading

Using `os.environ`

Using `os.getenv()`

Accessing variables in a `.env` file

Accessing the file using `configparser`

Accessing the file using `python-dotenv`