DEV Community: Shantanu Nighot

How to save Environment Variables in Python virtual env

Shantanu Nighot — Sun, 29 Jan 2023 06:45:21 +0000

Saving environment variables in a Python virtual environment can be a great way to keep your project's dependencies and settings organized and separate from other tasks on your machine.

I use Django to create APIs for the front end of my website. Django needs many credentials that should not be exposed to the public or in the GitHub repository. I have already created an article on How to hide Django SECRET_KEY on Public Repositories. But if we want to use it on a local machine then still have to replace the code with the original credentials.

Here are the steps to take to save environment variables in a Python virtual environment:

Create a virtual environment:

To create a virtual environment, you can use the virtualenv package. To install it, open a terminal and run pip install virtualenv. Once it is installed, you can create a new virtual environment by running a command virtualenv venv, where "venv" is the name of your virtual environment.

Activate the virtual environment:

To activate the virtual environment, run source myenv/bin/activate on Linux and macOS or source myenv\Scripts\activate on Windows. Once the virtual environment is active, the name of the virtual environment will appear in the command prompt.

Set environment variables:

To set an environment variable, you can use the export command on Linux and macOS or the set command on Windows. For example, to set an environment variable named DJANGO_SETTINGS_MODULE with the value myproject.settings, you can run export DJANGO_SETTINGS_MODULE=myproject.settings on Linux and macOS or set DJANGO_SETTINGS_MODULE=myproject.settings on Windows.

Keep in mind that the environment variables set through the command prompt are temporarily saved in the environment and will be destroyed once the environment is deactivated. Use the following method to set the variables automatically every time the environment is activated.

Save environment variables:

To save environment variables so they persist even after you close the terminal or deactivate the virtual environment, you need to add the export or set commands to a shell script.

On Linux and macOS, create a file named activate in the bin directory of your virtual environment with the following contents:

#!/bin/sh
export SECRET_KEY="django-insecure- *************"

On Windows, find the file named activate or create a file named activate in the Scripts directory of your virtual environment with the following contents:

set SECRET_KEY="django-insecure- *************"

Use saved environment variables:

We can use the saved variables using os.environ.get(). In my case, I want to use the saved secret key for my Django project. In settings.py file, I will import the OS module using import os. Then I will get the saved environment variable SECRET_KEY using the command os.environ.get('SECRET_KEY').

Note: Make sure you do not expose activate file in your GitHub repository. To ensure this add your env directory to .gitignore file. Directories and files included in .gitignore are not included while pushing the code to the repository.

Deactivate the virtual environment:

To deactivate the virtual environment, you can run the deactivate command. This will take you out of the virtual environment and return to the global Python installation on your machine.

By following these steps, you can successfully save environment variables in a Python virtual environment, keeping your project dependencies and settings organized and separate from other projects on your machine.

Please let me know if I am missing something in this tutorial or need help with any step.

Thanks for reading. See you in the next article.

A quick and easy alternative to Heroku is Railway.app

Shantanu Nighot — Sun, 16 Oct 2022 10:46:30 +0000

Suppose you are currently using Heroku to deploy your projects like Django apps. In that case, you should have received mail from Heroku that they are going to stop free dynos, free databases and all other free services currently provided. This was very sad, as Heroku became my favourite cloud platform last year. I chose Heroku over other cloud platforms because it provides services for free as well as the deployment process is very straightforward. You can deploy your project in very less configurations.

But now as Heroku is going to stop its free services, I had to search for another platform to deploy my existing as well as upcoming projects. At first, I explored famous platforms like Google Cloud Platform, Amazon Web Services and Microsoft Azure. But all of them provided free deployment for a limited time or limited credits.

After searching the Web, I found the Railway.app website and decided to give it a try after going through its features and free plan. A few features that I like personally are,

Quick and easy setup
Deploy from GitHub Repository
Chose Database of your choice
Automatic Deployments
Apps do not sleep like Heroku apps
Templates for production-ready applications
Configurations same as Heroku
Full access to Database unlike Heroku

The starter plan of the Railway.app includes $5.00 of usage each month with an execution time limit of 500 hours which I think is sufficient for small projects on your portfolio.

In this article, I will show you how to move your Django project from Heroku to Railway efficiently. My Octoprofile project is currently deployed on Heroku and I will move it to Railway.app.

Create new project and add Config variables

Sign up on Railway.app and go to your dashboard to create a new project. I will be deploying the project from the same GitHub repository used to deploy on Heroku. We are able to do this because configuration files like Proc file and runtime file required are the same for both platforms.

After connecting your GitHub account and selecting the respective repository, DO NOT Deploy directly because we are missing the Config variables like SECRET KEY and DATABASE URL which will fail the deployment. If your Project does not have any variables or Database connected then only you can click on Deploy Now. Click on Add variables to add variables from the Heroku dashboard.

To get the variables for your project go to the Heroku dashboard and you can find Config Vars in the settings of your app. Copy and paste all Key-value pairs to Your Railway project except for the DATABASE_URL. Because we are not going to use the Heroku Database. We will create a new Database for the Railway.app project.

Create a new Database and link it to your Project

To create a new database, click on the New button inside your project dashboard and click on Database. You can select any Database like PostgresSQL, Redis, MongoDB and MySQL. I will use PostgresSQL for this project.

Once it's created, go to connect tab and copy Postgres Connection URL. Now go to your project and create a new variable DATABASE_URL and paste copied link to its value.

Note that whenever you make any changes in the project like adding new variables, every time the Railway will auto-deploy the changes. And will fail many times until all configurations are in place and this is fine. You can always check the logs to know what went wrong.

Now, you should see something like this. The deployment is successful and you will get a link to your website.

Things to do after the successful deployment.

First thing is to change the default domain name provided by Railway and set it to something that makes sense. I will set it to octoprofile.up.railway.app. You can also set up a custom domain if you have one.

Now if you visit the URL, you may get Bad Request (400). This is because you might have set ALLOWED_HOST as ***.herokuapp.com in the** settings.py** file of the Django project. You will change it to the one provided by Railway.app.

Once you push changes to GitHub, Railway.app will deploy the changes and the site will appear on the link provided.

You can see my deployed site at octoprofile.up.railway.app. Enter your GitHub username and hit enter.

If you want to create a user just go to Postgres Database and add a new row in the auth_user table. Don't forget to give True for the is_superuser field else you will not able to login to the Admin page. Similarly, you can add other data into respective tables.

That is all to move from Heroku to Railway.app. Let me know if you want to add anything or if any steps are missed. Also, you can comment here if you get any errors while deploying so we can solve it together.

Thanks for reading. See you in the next article. 😊

Be better than yesterday's you.

An alternative method to view your Heroku Database tables

Shantanu Nighot — Sun, 09 Oct 2022 08:46:04 +0000

In this article, I will tell you about an alternative method to view your database from the Heroku platform. Check the tables of your Heroku Postgres Database used by your website or an application.

When I deployed my application on Heroku, I was curious about checking tables created by Django and Models. But was unable to check it as Heroku provides no specific functionality to view what's inside a database. After exploring, I found this application that helped me go through my database tables.

Following are the quick steps to get started,

Download a small application

Firstly download the application file respective to your machine from this link and extract the zip file.

Run the application

Run this application and this will start one localhost session at https://localhost:8081 or any other port.

Visit localhost

After visiting the respective localhost link, you will see the following page. You will get an option to connect with your database using a standard URI, Database credentials or SSH.

Get Database credentials

Visit your application dashboard on Heroku, go to the Resources tab and click on Heroku Postgres. On the Datastores page, go to the Settings tab and click on View Credentials. You will get the Hostname, Database, User, Port, Password, URI and Heroku CLI information.

Connect to Database

You can use these credentials to connect to your database. For this article, I will be using the URI method. Copy the URI from the Heroku dashboard, paste it into the Scheme tab and click on connect button. This will open your database and the tables in it. You can get a lot of information about your databases such as tables, rows and table structure. Find out now yourself by following the steps above.

Thank you for reading this article. You can also subscribe to my newsletter to receive my articles straight to your inbox.

Be better than Yesterday's you!

Let Cloudinary handle image uploads in your Django application.

Shantanu Nighot — Sun, 15 May 2022 12:17:48 +0000

Hi, everyone,

It's been a while since I uploaded my last article. I would love to tell you that I have joined Publicis Sapient six months ago as a Junior Associate Technology. And now, I have been promoted to Associate Technology L1. These first six months were so tough for me as I was new to this corporate world. I got exposed to technologies like Java, JavaScript, MongoDB, SQL and MySQL. Everything was so overwhelming, and I was unable to give time for blogging. But after spending six months working hard for the company, I decided to spare some time for my blog.

In this article, we are going to create a simple Django application called PhotoGallery where image uploads are handled by Cloudinary. The main focus of this article is to know how to integrate Cloudinary with Django models.

Cloudinary

The company provides cloud-based image and video management services. It enables users to upload, store, manage, manipulate, and deliver images and videos for websites and apps. Wikipedia

Overall, I have been using Cloudinary for a long time and would recommend it to developers who not only want to store images and videos but wants to optimise them for websites and apps.

We are going to create a Django application that displays the Images from the database and has a form to upload Images with its Alt name.

Pre-requisites

Django pip install Django
Cloudinary pip install cloudinary
Create an account on Cloudinary.com and note down the cloud name, API key and API secret key. We will need these to integrate the Cloudinary account with our application.

Create a Django project

If you are familiar with Django, these parts of the tutorial are piece of cake for you. Before you create a Django project, it is recommended to create a virtual environment to install dependencies for the project within the environment only. Check out the article A virtual environment in Powershell to know how to create virtual environments. Make sure to install Django using pip install Django.

Run the following command to create a Django project. I am using photogallery but you are free to use any name you want.

>> django-admin startproject photogallery

Create an app named gallery

Now that the project is created open the directory with the project name using cd photogallery. And create an app with the name gallery using the following command.

>> python manage.py startapp gallery

The project structure should look like this,

Add a new app to the INSTALLED_APP list in the settings.py file.

./photogallery/settings.py

INSTALLED_APPS = [
    .
    .
    'gallery',
]

Also, add cloudinary credentials we copied earlier from the website in the settings file.

./photogallery/settings.py

CLOUDINARY = {
  'cloud_name': 'your cloud name',  
  'api_key': 'your cloudinary API key',  
  'api_secret': 'your cloudinary API secret key',  
}

Create a Model to store photos

We will create a Model named Photo with fields like image, alt and upload date. But you are free to add a few extra fields like tags and image descriptions.

./gallery/models.py

from django.db import models
from cloudinary.models import CloudinaryField

class Photo(models.Model):
    image = CloudinaryField('image')
    alt = models.CharField(max_length=200)
    uploaded = models.DateTimeField(auto_now_add=True)

You should have noticed that we added CloudinaryField instead of Django's ImageField. It's a special Field provided by Cloudinary for uploading images and videos. The first argument is the type of file expected for upload. There is a lot more you can do with this field. Check out this documentation to know more about the cloudinary fields and arguments.

./gallery/admin.py

from gallery.models import Photo

admin.site.register(Photo)

Don't forget to run the following commands, these will create migrations and new tables in the database from our models.

>> python manage.py makemigrations
>> python manage.py migrate

To quickly check how our model is working, you can create a superuser to access the admin panel. Run the following command to create a superuser. It will ask you for your username, email, and password.

>> python manage.py createsuperuser

After creating a superuser, you can run command python manage.py runserver, access the admin panel at /admin and log in using these credentials.

Try to upload a few images and with successful submission of the form, you should see something like this.

You can see that the image is saved in link form and not the image directly. I will show you how to access this image in the template later in the tutorial.

Create a form to upload photos

Let's create a simple form to upload images directly from the homepage. forms.py is not created by default so you will need to create it in the app directory. You can do a lot more with the Django forms than we did in this tutorial. Check out Django documentation to know more about Django forms.

./gallery/forms.py

from django.forms import ModelForm      
from .models import Photo

class PhotoForm(ModelForm):
  class Meta:
      model = Photo
      fields = ['image', 'alt']

Create a class-based view

We are creating a class-based view that inherits Django's generic view called CreateView. We are using CreateView instead of ListView because we are passing objects as well as a form in the same view and using ListView does not allow us to render the form correctly. We have passed the Photo model, PhotoForm and a home.html template to the PhotoList view. Don't worry, we are going to create this simple template later in this tutorial.

./gallery/views.py

from django.urls import reverse
from gallery.models import Photo
from django.views.generic import CreateView
from gallery.forms import PhotoForm

class PhotoList(CreateView):
    model = Photo
    form_class = PhotoForm
    template_name = "gallery/home.html"

    def get_context_data(self, **kwargs):
        context = super(PhotoList, self).get_context_data(**kwargs)
        context['photos'] = Photo.objects.all().order_by('-uploaded')
        return context

    def get_success_url(self):
        return reverse('home')

We are passing extra objects created from our Photo model by calling the method get_context_data. Later you can access these objects in a template using the context name photos.

Add URL to homepage

In photogallery/urls.py add the following lines of code to connect our PhotoList view to the URL.

./photogallery/urls.py

from gallery.views import PhotoList

urlpatterns = [
    .
    path('', PhotoList.as_view(), name='home')
]

Create a template for the homepage

This is the last step of our tutorial. We need a template to show our photos and the PhotoForm to upload photos with alt names. We have already defined the template name in View as home.html so we will create the same in a new directory gallery/templates/gallery/home.html and add the following code.

./gallery/templates/gallery/home.html

<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta http-equiv="X-UA-Compatible" content="IE=edge">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Photo Gallery</title>
    <style>
        .grid-container {
          display: grid;
          grid-template-columns: auto auto auto auto;
          gap: 10px;
          background-color: #dadada;
          padding: 10px;
        }
    </style>
</head>
<body>
    <h1 style="text-align:center;">Photo gallery</h1>
    <div class="grid-container">
        {% for photo in photos %}
            <img src={{ photo.image.url }} alt="{{ photo.alt }}" style="width: 100%;">
        {% endfor %}
    </div>
    <br>
    <center>
        <h3>Upload your photos here.</h3>
        <form method="POST" action="" enctype="multipart/form-data">
            {% csrf_token %}
            {{ form }}
            <input type="submit" value="Submit">
        </form>
    </center>
</body>
</html>

When we created the PhotoList view, we passed an extra context photos which stores all Photo objects. That same context is referred in for loop to iterate through each object. Then each object has an image property which further has a URL property which points to the cloudinary cloud location.

After the photo gallery, we have a simple form to upload photos. And the final application looks like the following,

And that's it, but you can not and should not stop here especially if you are a beginner. Dig deeper, check out the Django documentation and add extra features and functionalities to your application. For example, you can add authentication required for forms to restrict others from uploading photos to your gallery. Check out the GitHub repository for the source code of this project.

https://github.com/magbanum/photogallery/

As I said earlier, the main focus of this article was to know how cloudinary can help us manage image uploads efficiently, hassle-free and without much integration process. All we needed is credentials and a cloudinaryField. And the bonus is that Cloudinary not only delivers the images but at the same time optimizes them. Which helps your application load faster.

I hope you found this article helpful. Please let me know if I did something wrong in this article or have any doubt regarding any step. I will be happy to help you with my knowledge and understanding.

Thanks for reading.🌻

Be better than yesterday's you!

All about python dictionary.

Shantanu Nighot — Sat, 09 Oct 2021 12:21:44 +0000

Python dictionary is one of the most used and powerful data structures in python. It is a collection of data in the form of key-value pairs. It is a feature-rich data structure used widely when working with structured data. Python dictionaries have many built-in methods that are used to access, update and delete the data from them which makes them easy to use. Python dictionaries and lists have similar properties but differ in how the data is accessed. Python lists use zero-based indexing whereas dictionaries use keys for referencing the data associated with it. Let's learn more about dictionaries and how to work efficiently with them.

Scope

The article explains the basic syntax and characteristics of python dictionaries.
The article includes basic CRUD operations with dictionaries.
The article has provided information about the built-in dictionary methods.

Do you know? Python dictionaries were unordered till Python version 3.6. After that, they were made ordered from version 3.7. Try running the following code on python version 3.6 and the latest one, and see the difference in results.
>> my_dict = {"name": "Shantanu Nighot", "age": 21, "grade": "A"}
>> print(my_dict)

They can be nested.

Python dictionaries just like lists can have another dictionary or list as a value.

>>> bagpack = {"Books": 5, "Camera": 1, "Fruits": {"Apple": 1, "Orange": 2}}
{"Books": 5, "Camera": 1, "Fruits": {"Apple": 1, "Orange": 2}}

Keys:

Keys must be unique.

Python dictionaries do not allow duplicate keys. This means any key is unique throughout the dictionary. If there are duplicate keys defined in the dictionary the last appeared key-value pair is considered only.

>>> mydict = {"name": "Shantanu", "age": 20, "grades": "A", "age": 22}
{'name': 'Shantanu', 'grades': 'A', 'age': 22}

You can see that the key “age” appeared multiple times with different values but only the last appearing value (22) is saved in a dictionary.

Keys must be immutable:

Keys in the dictionary can be of any immutable data type. For example string, number, boolean or binary data type can be used as a key in a python dictionary. Tuples being immutable can also be used as a dictionary key.

>>> mydict = {"name": "Shantanu", 21: "age", False: "Palash", (1,2): "numbers"}
{"name": "Shantanu", 21: "age", False: "Palash", (1,2): "numbers"}

Amazingly, we can also use the python function as a key.

>>> def name(): pass
>>> mydict = {name(): "Shantanu", False: "Palash"}
{name(): "Shantanu", False: "Palash"}
>>> mydict[name()]
Shantanu

You can use any of the above-stated data types or a function as a key. But you should have proper reason and logic behind using them, mainly in the case of tuples and functions.

Values

Unlike Keys, Values don't have any restrictions. They can be of any data type or even another dictionary or List. We can even use functions as a value associated with the dictionary key.

>>> def fruits(): return ["apple", "oranges"]
>>> mydict = {"name": "Shantanu", "things": fruits()}
>>> mydict["things"]
["apple", "oranges"]

When we access the function from the dictionary using its associated key, the function is executed and returns the value from it instead of the function itself. That's why, above code prints list ["apple", oranges"] and not fruits().

CRUD operations with a dictionary

Till now you got familiar with the dictionary, key, value and their properties. Now, it's time to see how dictionaries work. We will see basic CRUD (Create, Read, Update, Delete) operations on Dictionaries.

Creating a dictionary
Reading/Accessing a dictionary
Updating a dictionary
Deleting a dictionary

Creating dictionary

You are already familiar with the syntax of the dictionary and have already created it in previous examples. The dictionary can be created using one or many key-value pairs separated by a comma and enclosed inside curly braces. Each key and its associated value is separated by a colon.

>>> my_dict = {"name": "Shantanu Nighot", "age": 21, "grade": "A" , "pass": True }
{"name": "Shantanu Nighot", "age": 21, "grade": "A" , "pass": True }
>>> type(my_dict)
<class 'dict'>

It's not necessary to create a dictionary with some data. We can also define an empty dictionary by either using empty curly braces {} or the dict() keyword.

>>> dict1 = {}
{}
>>> dict2 = dict{}
{}
>>> print(type(dict1), type(dict2))
<class 'dict'> <class 'dict'>
>>> print(len(dict1), len(dict2))
0 0

Reading/Accessing dictionary

We can not use numerical indices in dictionaries as we do in lists. And if we do so, Python will raise an exception KeyError as it can not find a key in the dictionary.

>>> mydict = {"fruit": "Mango"}
>>> mydict[0]
Traceback (most recent call last):
  File "<pyshell#13>", line 1, in <module>
    mydict[0]
KeyError: 0

In python dictionaries, instead of indices, keys are used to access values associated with them.

>>> my_dict = {"name": "Shantanu Nighot", "age": 21, "grade": "A" , "pass": True }
>>> my_dict["name"]
Shantanu Nighot

From the nested dictionary, values of sublists or subdictionaries can be accessed using an extra index or a key just like in the following example.

>>> fav_list = {"colors": ["Skyblue", "grey"], "number": 21, "fruits": ["Orange", "Mango", "Blueberries"]}
>>> fav_list["fruits"][1]
Mango
>>> fav_list["fruits"][0:2]
['Orange', 'Mango']

There are few built-in dictionary methods like items(), keys(), values(), and get(). They can also be used for accessing the data from the dictionary. We are going to learn about them later in this article.

Now as you got familiar with accessing the data from the dictionaries, you can start working on using them with Python's operators and built-in functions.

>>> numbers = { "num1": 10, "num2": 13}
{ "num1": 10, "num2": 13}

>>> numbers["num1"] + numbers["num2"]
23

>>> "num2" in numbers
True

>>> sorted(mydict, reverse=True)
{ "num2": 13, "num1": 10}

Updating dictionary

Any value in the dictionary can be changed by referencing its key.

>>> numbers = { "num1": 10, "num2": 13}
>>> numbers["num1"] = 12
>>> numbers
{ "num1": 12, "num2": 13}

New data can be added using the same above syntax. If the key is present in the dictionary then Python reassigns a new value to a key and if a key is not found then it creates a new key and assigns value to it.

>>> numbers = { "num1": 10, "num2": 13}
>>> numbers["num3"] = 12
>>> numbers
{ "num1": 10, "num2": 13, "num3": 12}

There is a built-in dictionary method update() which is also used to update the data from a dictionary. We will learn about it later in this article.

Deleting dictionary

The del function can be used to delete the whole dictionary. After that, any reference to it will throw a NameError.

>>> numbers = { "num1": 10, "num2": 13}
>>> del(numbers)
>>> numbers["num1"]
File "main.py", line 3, in <module>
    numbers["num1"]
NameError: name 'fav_list' is not defined

If you do not want to delete the dictionary and just want to delete data in it, then you can assign an empty dictionary to it.

>>> numbers = { "num1": 10, "num2": 13}
{ "num1": 10, "num2": 13}

>>> numbers = {}
{}

There are few built-in dictionary methods used to perform deleting operations on dictionaries. We are going to learn about them in the next topic.

Built-in dictionary methods

items()
keys()
values()
get()
update()
clear()
pop()
popitem()

.items()

This method returns the list of tuples containing key-value pairs from a dictionary. The first element of each tuple is the key and the second element is the value associated with that key.

>>> numbers = { "num1": 10, "num2": 13, "num3": 23}
>>> numbers.items()
dict_items([('num2', 13), ('num3', 23), ('num1', 10)])
>>> type(numbers.items())
<class 'dict_items'>

You can see that the method returns the data of type "dict_items". Hence, we need to convert it into a list so that we can access them using normal indexing.

>>> numbers = { "num1": 10, "num2": 13, "num3": 23}
>>> list(numbers.items())
[('num2', 13), ('num3', 23), ('num1', 10)]

>>> list(numbers.items())[0]
('num2', 13)
>>> list(numbers.items())[1][1]
13

Mostly, the items() method is used while working with loops and dictionaries.

>>> numbers = { "num1": 10, "num2": 13, "num3": 23}
>>> for k, v in numbers.items(): print(k, v)
num2 13
num3 23
num1 10

.keys()

This method returns the list of all keys in the dictionary. The return type is "dict_keys", hence we need to convert it into a list using list() so that we can use normal indexing.

>>> numbers = { "num1": 10, "num2": 13, "num3": 23}
>>> list(numbers.keys())
['num1', 'num2', 'num3']

>>> for key in numbers.keys(): print(numbers[key])
10
13
23

.values()

This method returns the list of all values in the dictionary. The return type is "dict_values", hence we need to convert it into a list for using normal indexing.

>>> numbers = { "num1": 10, "num2": 13, "num3": 23}
>>> list(numbers.values())
[10, 13, 23]

.get(key, default_value)

This method returns the value associated with the key if it's present in the dictionary else returns None. This method does not raise KeyError if the key is not found in the dictionary and returns the default value which is None by default.

>>> numbers = { "num1": 10, "num2": 13, "num3": 23}
>>> numbers.get("num3")
23
>>> numbers.get("num4")
None

The second argument is optional but can be very helpful when we want to return a specific value when the key is not found.

>>> numbers = { "num1": 10, "num2": 13, "num3": 23}
>>> numbers.get("num5", 0)
0

.update()

This method is used to update existing data and to add new entries to the dictionary. There are several ways of using the update() method.

>>> numbers = { "num1": 10, "num2": 13, "num3": 23}
>>> numbers.update({"num2": 5, "num4": 10})
>>> numbers
{'num1': 10, 'num2': 5, 'num3': 23, 'num4': 10}

>>> numbers.update([("num5", 4), ("num2", 11)])
>>> numbers
{'num1': 10, 'num2': 11, 'num3': 23, 'num4': 10, 'num5': 4}

>>> numbers.update(num1=0, num6=2)
>>> numbers
{'num1': 0, 'num2': 11, 'num3': 23, 'num4': 10, 'num5': 4, 'num6': 2}

.clear()

This method is used to remove all key-value pairs from the dictionary.

>>> numbers = { "num1": 10, "num2": 13, "num3": 23}
>>> numbers.clear()
>>> numbers
{}

.pop(key, default_value)

This method returns the value associated with the key and removes the key-value pair from the dictionary.

>>> numbers = { "num1": 10, "num2": 13, "num3": 23}
>>> numbers.pop("num2")
13
>>> numbers
{ "num1": 10, "num3": 23}

If the key is not present in the dictionary then python raises an exception KeyError. Hence, it is recommended to provide default value as a second argument. This default value will be returned if the key is not found in the dictionary.

>>> numbers = { "num1": 10, "num2": 13, "num3": 23}
>>> numbers.pop("num4", 0)
0
>>> numbers
{ "num1": 10, "num2": 13, "num3": 23}

.popitem()

This method returns the last key-value pair as a tuple and removes it from the dictionary. If the dictionary is empty, it raises an exception KeyError.

>>> numbers = { "num1": 10, "num2": 13, "num3": 23}
>>> numbers.popitem()
('num3', 23)
>>> numbers
{ "num1": 10, "num2": 13}

Summary

Python dictionary is the collection where data is store in the form of key-value pairs.
Python dictionaries are immutable, ordered and can be nested.
Keys must be of immutable data type whereas values can be of any data type.
Functions and few python keywords can also be used as keys and values.
Few built-in dictionary methods make it easier to work with dictionaries.

I have also created a Jupyter notebook for this article on GitHub. This notebook will help you run the above codes effectively. Check it out by clicking here.

I hope you found this article helpful. Do comment below if I missed something to make this thread more informative.

You can also sign up for the newsletter to get my articles directly in your inbox.

See you in the next article.

Be better than yesterday's you.

Python program to make text URL safe.

Shantanu Nighot — Thu, 09 Sep 2021 07:24:39 +0000

Liquid syntax error: Variable '{{% raw %}' was not properly terminated with regexp: /\}\}/

Starting Django project with MongoDB using Djongo.

Shantanu Nighot — Thu, 22 Jul 2021 05:55:06 +0000

Hi everyone, as an enthusiast developer, I wanted to learn about databases. There are lots of them on the internet but the one that caught my attention is MongoDB. Recently I heard about the unique database called MongoDB. It's a NoSQL document database where data is stored in documents instead of rows and columns as in SQL databases.

After researching more about MongoDB, its features and applications, I started learning about it and also completed my first Basic course at MongoDB University. Thanks to GitHub's Student Developer Pack and MongoDB to provide free access to MongoDB University courses.

https://www.linkedin.com/posts/shantanu-nighot_my-mongodb-university-course-proof-of-completion-activity-6820338711700520960-50C1

With basic knowledge of the MongoDB database, I decided to use it with one of my projects and here is what I have built.

Quotes gen project: https://quotes-gen-project.herokuapp.com

In this project, I have created a model to store quotes and authors in a MongoDB database and View to display a random quote from a collection of more than 2 lacs of quotes.

So, I am writing this article to share with you the steps I followed to connect the MongoDB database with my Django project.

There are currently 3 ways to connect the MongoDB database to Django, using

PyMongo
MongoEngine
Djongo [Dj ango + M ongo]

I personally loved how Djongo manages to connect MongoDB to Django with minimum changes. I will suggest using Djongo for beginners as it does not change the original Django ORM and everything is kept minimal and simple. And in this article, we are going to use it to connect Django with MongoDB.

Pre-requisites

Before proceeding further make sure you have basic knowledge of how things work in Django and MongoDB. I have mentioned few resources to get started with them.

Let's get started,

Step 1: Virtual environment setup

Make sure you have created a virtual environment and activates it using the following commands,

# Install virtualenv package
python -m pip install --user virtualenv

# Create a virtual environment with the name 'env'
python -m venv env

# Activate the virtual environment
env\Scripts\activate

Step 2: Install packages

Install the required packages like Django and Djongo using the following command,

pip install Django Djongo pymongo[srv]

Step 3: Start the Django project

Now as our virtual env is activated, let's create our Django project and the app.

# Start the Django project
django-admin startproject <project-name>

# Open the project folder
cd <project-name>

# Create an app
python manage.py startapp <app-name>

# Start the server to see the project running
python manage.py runserver

Don't forget to add the newly created app to INSTALLED_APPS in the settings.py file.

Step 4: MongoDB cluster creation

Now our app is ready, let's head to MongoDB to create our Atlas cluster.

Go to https://cloud.mongodb.com and sign in
Create an Organisation and a Project
Create Cluster
- Click on Create a Cluster
- Select the Shared Database deployment for this tutorial
- Change the Cluster name
- Click on Create button (Takes few minutes to complete the process)
Click on Connect and Create a Database User with Username and Password. If it asks to add an IP address then Click on Add IP address without adding anything. This allows access to your cluster from any IP address and is not recommended when creating a real-life project.
Then select the connection type Connect your application and copy the provided link. We will need it later.

For more information, visit the MongoDB Documentation, Get started with Atlas.

Step 5: Database information in Django

Everything is ready to be served. In settings.py file add the following information in DATABASE and replace the credentials like username, password, and atlas cluster with yours. You can add the database name of your choice, then Django will create that database on migration. Add admin instead of <myFirstDatabase>.Remember the link we copied earlier, you can also use it in place of the given link.

DATABASES = {
        'default': {
            'ENGINE': 'djongo',
            'NAME': 'your-db-name',
            'ENFORCE_SCHEMA': False,
            'CLIENT': {
                'host': 'mongodb+srv://<username>:<password>@<atlas cluster>/<myFirstDatabase>?retryWrites=true&w=majority'
            }  
        }
}

I may have skipped some Django parts to focus this article on MongoDB connection. Please make sure you have followed all steps required to run a Django project.

Step 6: Where the magic happens...

It's time to actually connect our Django project to a MongoDB cluster. Run the following commands to make migrations and migrate the database.

# Make migrations
python manage.py makemigrations

# Migrate
python manage.py migrate

And that's it. You can see in your MongoDB cluster that the new database is created with the name provided by you and few collections inside that Database like the following,

If you get an error djongo.database.DatabaseError while migrating the database, run the following command pip install sqlparse==0.2.4 to fix the issues.

Step 7: Creating Models

To know how Django models are connected to the MongoDB database, let's create a model in our project. For example, in the following code, the model People is created with Name, Age fields.

class People(models.Model):
    Name = models.CharField(max_length=100, default=None)
    Age = models.IntegerField(default=None)

Don't forget to register the model by adding the following lines in the admin.py file.

from . models import People
admin.site.register(People)

Again run python manage.py makemigrations and python manage.py migrate to migrate the database.

In the MongoDB cluster, you can see that the new collection testapp_people is created. In Django, the models are connected to the MongoDB collections with the name <app-name>_<model-name> where app-name is the name of Django app and model-name is the name of the model inside that app.

Step 8: Adding data

Last but not the least, let's create a superuser using the following command and add some data using Django admin.

python manage.py createsuperuser

# To start the server
python manage.py runserver

After adding some data from the admin panel, you can see that the collection is crowded with some data which are called Documents. MongoDB stores the data in the form of documents.

This is all you need to know while starting a Django project with MongoDB. Now go ahead and create some interesting projects. Check out GitHub repository for the project I mentioned in the start to help you in the process.

If you face any issues in the process, don't hesitate to mention them in a comment. I am always peeping at my inbox for new comments and will get back to you immediately.

If you have any suggestions or feedback related to my articles, please let me know.

Thank you for reading this article. I hope this was helpful to you.🌻

Octoprofile - the Django project

Shantanu Nighot — Mon, 28 Jun 2021 13:47:13 +0000

I am very happy to share with you the Django project I have been working on last 20 days, it's called an Octoprofile. Project Octoprofile displays the GitHub profile in a better way with Charts and sortable lists.

Octoprofile collects the user information on GitHub using GitHub API endpoints and then Django's function-based view passes it to the template after manipulating the data.

To view your Octoprofile visit octoprofile.herokuapp.com and enter your GitHub username. Don't have a GitHub profile, check mine by typing magbanum or by clicking here. The website might be slower for the first load as the project is deployed on Heroku's free dyno.

I have worked a lot on the coding and researching part as I had to make everything from scratch. As I didn't know JavaScript I faced problems in the beginning while sorting the list and displaying the charts. But with the basics of JavaScript and few JavaScript libraries (ChartJS & ListJS), I was able to make them all work.

About the project

I have created an app inside the Django project called profiles and created two templates home.html for the landing page and profile_page.html for the profile page.

The function-based views get_username and get_repodata are used to collect and manipulate the data and pass it to the template profile_page.html. The class-based API views TopLanguages, MostStarred, and StarsPerLanguages are used to create API endpoints for top languages by the user, most starred repositories of the user, and stars per languages respectively. Check out GitHub repository for source code.

ChartJS uses these endpoints and displays charts accordingly. The project is then deployed on the Heroku (free dyno) using Heroku CLI.

Few project features.

Clean UI
Responsive design
Easy navigation
Graphical view

Coded with

Python
JavaScript
HTML5
CSS3

Built with

Django High-level Python Web framework
Django REST framework Powerful and flexible toolkit for building Web APIs.
ChartJS Simple yet flexible JavaScript charting for designers & developers
ListJS Perfect library for adding search, sort, filters to tables, lists and various HTML elements.
GitHub API Create calls to get the data you need to integrate with GitHub.
Jquery Fast, small, and feature-rich JavaScript library.

Deployed on

Heroku Platform as a service (PaaS) that enables developers to build, run, and operate applications entirely in the cloud.

Overall this project was really amazing experience and taught me a lot of things in the development process. I will love to see you acknowledge my efforts by visiting octoprofile.herokuapp.com.

How's that?

Any suggestions and feedbacks are always welcomed. Bugs are not welcomed but you can definitely let me know if you found one. Visit the project repository here and give it a ⭐ if you liked your Octoprofile and share it with everyone.

Click here to share my project on Twitter.

Thank you for taking the time to read this article.🌻

Be better than yesterday's YOU!

Pagination in Django.

Shantanu Nighot — Wed, 23 Jun 2021 15:33:54 +0000

Pagination is the process of splitting the contents of a website, or a section of contents from a website, into discrete pages. One should use pagination to make his/her website more efficient to use when working with a huge list of contents for example "List of 100 blog posts". Also, paginating the content helps websites from being overcrowded with the content and make visitors focus on small content at a time. This increases the readability and visitors are likely to return to your website again. Django has a pre-built class called "Paginator" to create and manage the paginated data. It can be imported by the following code.

from django.core.paginator import Paginator

In Django, views are of two types which are Function-based views and Class-based views. Both types of views have different ways of paginating the content but the overall working is the same.

For this tutorial, we will be using my project called "One for all Blogging platform". For the full source code of my project visit the oneforall-blog GitHub repository.

https://github.com/magbanum/oneforall-blog/

In this project, I have created an app named blog which has the function-based view "newsview" with the following code,

blog/views.py

def newsview(request):
    blogs = get_news()
    return render(request, 'blog/news_posts.html',{'blogs':blogs})

The newsview function collects data from another function and passes it to template named news_posts.html and then template displays that data. The HTML code looks like this.

blog/template/blog/news_posts.html

 {% for blog in blogs %}
  <div class="col">
        ....
        <h5 class="mb-1">{{ blog.title }}</h5>
        ....
  </div>
{% endfor %}

Currently, the newsview and the news_posts.html template displays all the items stored in the blogs on a single page. So we need to add some code to paginate the items.

Let's first import the Paginator using the following line of code.

from django.core.paginator import Paginator

In function-based views, we need to specify the data which we want to paginate and the number of items by which we want to paginate the data. For the newsview we want to paginate the contents of blogs by 6 items. Also, in the previous code, we returned the blogs to the template but now we will need to return paginated data that is page_obj. The final code will become,

Project directory: blog/views.py

from django.core.paginator import Paginator

def newsview(request):
    blogs = get_news()
    paginator = Paginator(blogs, 6)
    page_number = request.GET.get('page')
    page_obj = paginator.get_page(page_number)

    return render(request, 'blog/news_posts.html',{'page_obj':page_obj})

Now that we have paginated the data successfully we need to display it in our news_posts.htmltemplate. Also, we have to add a navigation panel for the visitors to navigate between the pages. But not to worry, there are few lines of code to include simple navigation with the links for the first, previous, next and last page. So the final template will look like this,

Project directory: blog/template/blog/news_posts.html

{% for blog in page_obj %}
  <div class="col">
        ....
        <h5 class="mb-1">{{ blog.title }}</h5>
        ....
  </div>
{% endfor %}

<!-- For page navigation links -->
<div class="pagination" style="width: fit-content;">
    <span class="step-links">
    {% if page_obj.has_previous %}
        <a href="?page=1" class="page-link" style="display: inline;">&laquo; first</a>
        <a href="?page={{ page_obj.previous_page_number }}" class="page-link" style="display: inline;">previous</a>
    {% endif %}

    <span class="current">
        Page {{ page_obj.number }} of {{ page_obj.paginator.num_pages }}.
    </span>

    {% if page_obj.has_next %}
        <a href="?page={{ page_obj.next_page_number }}" class="page-link" style="display: inline;">next</a>
        <a href="?page={{ page_obj.paginator.num_pages }}" class="page-link" style="display: inline;">last &raquo;</a>
    {% endif %}
    </span>
</div>

And that's it. After running the server you will get nicely organised content with the navigation links to move between the pages.

In the case of Class-based Listview, we don't need to import the Paginator as ListView has its own way of paginating the List. All we have to do is specify the number by which we want to paginate the list. The project "One for all Blogging system" also have a class-based view named HomeView which is paginated by 6 items.

Project directory: blog/views.py

from django.views.generic import ListView

class HomeView(ListView):
    paginate_by = 6
    model = Post
    template_name = 'blog/home.html'
    ordering = ['-post_date']

The remaining steps are the same as in the function-based view. We can add navigation links just like we did with our newsview. You can see the code for template home.htmlhere.

Visit the One for all Blogging platform at https://oneforall-blog.herokuapp.com/ to see the working of our code.

And that's all about the Pagination in Django. For the full source code of my project visit the GitHub repo here and give a star if you like my work.

I hope this article was helpful for you. Please do subscribe to the newsletter to get my blog posts directly in your mailbox📬.

A virtual environment in Powershell.

Shantanu Nighot — Mon, 31 May 2021 06:54:43 +0000

The virtual environment is used to create an isolated environment for the different projects. This helps developers to keep the dependencies required by projects separate. So let's learn basic commands to create and activate the virtual env and install packages on it.

Prerequisites,

Python
PIP

Install virtualenv package using the following command,

python -m pip install --user virtualenv

Now go to the folder where you want to create a virtual environment and run the following command to create a virtual environment with the name "env". This will create the folder named "env" in a directory.

python -m venv env

You can name it whatever you what just make it logical and relative to your project.

To activate the virtual environment, go to the directory where you installed it and run the following command by replacing "env" with your environment name.

env\Scripts\activate

To install any package make sure the virtual environment is active. Use the following command to install the package.

pip install package-name

To install multiple packages use a comma , to separate package names. For example,

pip install requests, Django, BeautifulSoup

To install packages using requirements.txt file, use the following command

pip install -r requirements.txt

To get package names installed on a virtual environment into a requirements.txt file use the following command.

pip freeze > requirements.txt

To deactivate the env use,

deactivate

I am a total beginner and might have missed command that I never used before. So, please comment below few commands that are helpful for developers. Thanks for reading and do follow for more such articles.

Never stop learning!

How to use Hashnode APIs in Python.

Shantanu Nighot — Mon, 17 May 2021 09:58:39 +0000

The Hashnode GraphQL APIs public beta was released in 2019 to allow the users to create apps based on it. Even if it was minimal release, it has a lot of queries and mutations that can be used.

Today we are going to learn how to use those Hashnode API queries in python to get the most out of it.

Steps for creating a python program to get the user details with Hashnode GraphQL API:

Create query in GraphQL
Add queries and headers in the program
Create a function to run the JSON Query
Get the output as JSON Dictionary

Some prerequisites are,

Make sure you have the Requests module installed on your system using pip install requests.
You will need to create/revote the Personal Access Token for the Authentication purpose. You can find it at Hashnode Developer settings.

Personal Access Token is like a private key to your Hashnode account. You can use a personal access token to interact with your Hashnode account using our APIs. Never disclose your access tokens and always keep them private. You can revoke the generated tokens at any time. @hashnode

Creating queries in GraphQL

In this tutorial, we are going to get user details like name,tagline,numFollowers,publicationDomain, etc. So we will need to create a query for those in GraphQL.

query {
  user(username: "magbanum") {
    username
    name
    tagline
    numFollowers
    publicationDomain
  }
}

You can add more properties like numFollowings,numReactions, location,photo,coverImage with the help of DOCS and SCHEMA available at Hashnode GraphQL API.

For now, let's stick to this query. You can replace "magbanum" with your own username to get personalised results.

Visit https://api.hashnode.com/ and paste the above query there and click the play button to run the script. The output will look like this,

Adding Query and Headers in Python program

You have now created your first query in GraphQL, let's use it in our python program.

Create a Python file named "hashnode-user.py", you can name it whatever you want but let's stick to "hashnode-user.py" for this tutorial.

Firstly import the requests module, by adding the following line,

import requests

Remember the Personal Access Token which you created earlier in prerequisites, we will need it now. Authorization header with the Personal access token is needed to run the queries or mutations hence we will pass them to the variable named headers.

headers = {
    "Authorization": "your_personal_access_token"
}

Create the Variable named query to store the query that we created in GraphQL as a multi-line string.

query = """{
  user(username: "magbanum") {
    username
    name
    tagline
    numFollowers
    publicationDomain
  }
}"""

Function to run JSON Query

We have now reached the final step of our tutorial. As of now we have created a query and added it to our program. In this step, we will create a function that takes query as an argument and returns the output in JSON format.

Create a function named run_query which takes the query and the headers we created as arguments.
Now post a request in a variable named response with the Hashnode API URL, query and headers as parameters.
If ... else statement is used to return the JSON output if status code is 200 else raise the exception with status code. Learn more about status codes in HTTP requests here.

The function will look like this,

def run_query(query, headers):
    response = requests.post(url="https://api.hashnode.com", json={'query': query}, headers=headers)
    if response.status_code == 200:
        return response.json()
    else:
        raise Exception("Query failed to run by returning code of {}. {}".format(response.status_code, query))

Also, we need to call our function and get the data from it. Add few lines in the program but outside the function.

result = run_query(query, headers)
print(result)

And we are all done. The final code will look like this,

import requests

headers = {
    "Authorization": "your_personal_access_token"
}

def run_query(query, headers):
    response = requests.post(url="https://api.hashnode.com", json={'query': query}, headers=headers)
    if response.status_code == 200:
        return response.json()
    else:
        raise Exception("Query failed to run by returning code of {}.".format(response.status_code))

query = """{
    user(username: "magbanum"){
        username
        name
        tagline
        numFollowers
        publicationDomain
    }
}"""

result = run_query(query,headers)
print(result)

now let's run the code.

>>> python hashnode-user.py
{'data': {'user': {'username': 'magbanum', 'name': 'Shantanu Nighot', 'tagline': 'Electrical engineer | Self-taught developer | Hashnode blogger', 'numFollowers': 2, 'publicationDomain': 'magbanum.tech'}}}

And look at that, we have just used Hashnode API to collect the information from the Hashnode profile. You can see that the data is in form of a dictionary and what if we don't want the whole dictionary but the part of it. We can do that too. This result dictionary works the same as the Python dictionary so we will be able to index using its keys.

For example, if we want to get the tagline of a user, we just need to index the "result" dictionary as result["data"]["user"]["tagline"]. Replace the last line in the code with the following,

print(result["data"]["user"]["tagline"])

See the result, you will get your own tagline if you replace "magbanum" with your username.

>>> python hashnode-user.py
Electrical engineer | Self-taught developer | Hashnode blogger

This look really clean and personalized. Try printing out bunch of other things like name, publicationDomain, and numFollowers.

We have successfully created a query in GraphQL, used it in our Python program, created the function to run the query and got the expected results. This concludes the tutorial, and I hope you were able to use Hashnode APIs in your python programs.

As a next step, you can add user Input to get the username for more personalized results. Visit the Github repo for this code.

I hope the above information was helpful to you. Please comment if you are facing trouble with the above code, I will try my best to fix it. And follow me on Hashnode for more such tutorials.

How to hide Django SECRET_KEY on Public Repositories.

Shantanu Nighot — Sat, 08 May 2021 13:57:55 +0000

While working on the Django project you may have seen variables like SECRET_KEY and other DATABASE-related information which is considered sensitive. When uploading the source code of deployed Django project on the internet, these data should be properly managed to avoid any misuse. But removing them from repo every time you push your data can become a hectic job and can cause errors in production.

Let's consider my Django project named " quotes-gen-project " deployed on Heroku. It shows the random quote from the database with its author name and also allows visitors to add more quotes. This is my first Django project so I kept it as simple as possible.

You can visit the Github repo for the source code of this project.

After Deploying the project on Heroku, you will have the following variables in the settings.py file.

SECRET_KEY = 'your-django-secret-key'

DATABASES = {
      'default': {
        'ENGINE': 'your-database-engine name',
        'NAME': 'database-name',
        'USER': 'database-username',
        'PASSWORD': 'database-password',
        'HOST': 'database-host'
        'PORT': '5432',
    }
}

Now follow the below steps to add the above environment variables in our Heroku config vars.

Login to your Heroku account
Select your Heroku app
Go to settings
And click on Reveal Config vars.

Here you need to add the key and value for the variables you want to add. For example, add SECRET_KEY in key and your-django-secret-key in value without quotes. Do this for all other variables like NAME, USER, PASSWORD, and HOST.

You have now added config vars in the app and need to do some changes to address them from our code.

Go to the settings.py file and do the following changes.

SECRET_KEY = os.getenv('SECRET_KEY')

DATABASES = {
      'default': {
        'ENGINE': 'django.db.backends.postgresql',
        'NAME': os.getenv('NAME'),
        'USER': os.getenv('USER'),
        'PASSWORD': os.getenv('PASSWORD'),
        'HOST': os.getenv('HOST'),
        'PORT': '5432',
    }
}

And that's it.

Now run the following commands in the command prompt in your project root directory and check that everything is working as before.

git add -A
git commit -m "commit messege"
git push heroku master
heroku open

Hurray, you were able to hide sensitive data from source code and can now share your work with the world without any worries.

Thanks for checking out this blog. I hope this information was helpful for you. Let me know by commenting below.

DEV Community: Shantanu Nighot

How to save Environment Variables in Python virtual env

Create a virtual environment:

Activate the virtual environment:

Set environment variables:

Save environment variables:

Use saved environment variables:

Deactivate the virtual environment:

A quick and easy alternative to Heroku is Railway.app

Create new project and add Config variables

Create a new Database and link it to your Project

Things to do after the successful deployment.

An alternative method to view your Heroku Database tables

Download a small application

Run the application

Visit localhost

Get Database credentials

Connect to Database

Let Cloudinary handle image uploads in your Django application.

Cloudinary

Pre-requisites

Create a Django project

Create an app named gallery

Create a Model to store photos

Create a form to upload photos

Create a class-based view

Add URL to homepage

Create a template for the homepage

All about python dictionary.

Scope

Contents:

What is a Python dictionary?

Properties of Dictionary:

They are mutable.

They are ordered.

They can be nested.

Keys:

Keys must be unique.

Keys must be immutable:

Values

CRUD operations with a dictionary

Creating dictionary

Reading/Accessing dictionary

Updating dictionary

Deleting dictionary

Built-in dictionary methods

.items()

.keys()

.values()

.get(key, default_value)

.update()

.clear()

.pop(key, default_value)

.popitem()

Summary

Python program to make text URL safe.

Starting Django project with MongoDB using Djongo.

Pre-requisites

Step 1: Virtual environment setup

Step 2: Install packages

Step 3: Start the Django project

Step 4: MongoDB cluster creation

Step 5: Database information in Django

Step 6: Where the magic happens...

Step 7: Creating Models

Step 8: Adding data

Octoprofile - the Django project

About the project

Few project features.

Coded with

Built with

Deployed on

Pagination in Django.

A virtual environment in Powershell.

Prerequisites,

How to use Hashnode APIs in Python.

Creating queries in GraphQL

Adding Query and Headers in Python program

Function to run JSON Query

How to hide Django SECRET_KEY on Public Repositories.