DEV Community: Nikita Kholin

The easiest way to send emails with Django (using SES from AWS)

Nikita Kholin — Fri, 08 Mar 2019 20:42:45 +0000

Do you want to send emails in your application? Notifications, newsletters, promotions, there are so many reasons to send emails.

After reading this article you so be able to send any sorts of emails from your Django application.

There are many ways you can send emails using different email sending services. But in this guide, I'll cover one of the most popular services, SES, or Simple Email Service. It has been developed by Amazon and can be used for free for up to 62000 emails per month.

To work with SES we’ll use django-ses library. It is very easy to set up and use.

But before we can start we need to...

Sign up on AWS

If you don't already have an AWS account you need to get one. You simply won't be able to use SES without it.

To sign up you need to go to the AWS sign up page.

Setting up

I’m assuming you already have a Django project. To install django-ses library you need to run the following command

pip install django-ses

Once the library is installed you need to tell Django to use this library as email backend by adding the following line to your settings.py

EMAIL_BACKEND = 'django_ses.SESBackend'

Then you need to get credentials for SES

Go to AWS
Choose IAM service
Add user
Choose a name for the user and “Programmatic access” type (to get access key ID and secret access key)
Attach AmazonSESFullAccess permission
Create user

After that, you should see “Access key ID” and “Secret access key”. Add them to settings.py like that

AWS_ACCESS_KEY_ID = 'YOUR-ACCESS-KEY-ID'
AWS_SECRET_ACCESS_KEY = 'YOUR-SECRET-ACCESS-KEY'

Sending an email

Now that we have all the credentials in place, we can send an email. We can do this using Django core functionality (django.core.mail.send_mail to be more specific).

from django.core.mail import send_mail
send_mail(
    'Subject here',
    'Here is the message.',
    'from@example.com',
    ['to@example.com']
)

And this email should be sent using SES.

So let’s jump into Django’s shell and try it out.

python manage.py shell
>>> from django.core.mail import send_mail
>>> send_mail(
...    'Subject here',
...    'Here is the message.',
...    'from@example.com',
...    ['to@example.com']
...)
boto.ses.exceptions.SESAddressNotVerifiedError: SESAddressNotVerifiedError: 400 Email address is not verified.
<ErrorResponse xmlns="http://ses.amazonaws.com/doc/2010-12-01/">
  <Error>
    <Type>Sender</Type>
    <Code>MessageRejected</Code>
    <Message>Email address is not verified. The following identities failed the check in region US-EAST-1: from@example.com</Message>
  </Error>
  <RequestId>3966220a-3c6c-11e9-909d-889357e40867</RequestId>
</ErrorResponse>

Oh no! We got an error… Who would’ve thought that we couldn’t send emails from from@example.com. Let’s fix this.

Sending emails from your personal email address

To begin with, we can send an email from your personal email address. This should be pretty acceptable if you’re building an MVP.

If you don’t want for everyone to see your personal email, you can create a separate email address for your application like your.application@gmail.com. If people will start blocking your email for some reason, it will not affect your personal email health.

To send emails from your email address you need to

Go to AWS
Choose SES service
Go to “Email Addresses" tab
Click "Verify a New Email Address"
Enter your email address of choice
Go to the inbox for the email you’ve specified
Confirm that you want to use your email in SES

Once you’re done, you should be able to send an email from the Django shell

>>>send_mail(
...    'Subject here',
...    'Here is the message.',
...    'your.application@gmail.com',
...    ['your.email.address+to.test@example.com']
...)
1

After that, you should receive an email

You might’ve noticed a number that has been returned by the send_mail method. This is the number of successfully sent emails, which in our case was 1.

If we try to send an email to some address and it will fail then the failed email will not get counted

>>> send_mail(
...     'Subject here',
...     'Here is the message.',
...     'your.application@gmail.com',
...     ['your.email.address+to.test@example.com', 'not.existing.email.for.real.asdvsrg@gmail.com']
... )
1

This send_mail call returned 1 because it could not send an email to the second address (not.existing.email.for.real.asdvsrg@gmail.com) because it does not exist. I just made it up.

Sending emails from a custom domain

Once your MVP is working and you have a domain for your application it's time to send emails from this domain. To do this you need to add this domain to SES

Go to AWS
Choose SES service
Go to the "Domains" tab
Click “Verify a new domain"

If you already have your domain on AWS Route53 then you just click through and your domain will be ready.

If you have your domain set up somewhere else you need to follow specified instructions.

Once you do follow the instructions correctly, you should see green verified status and the domain should be ready for sending emails via SES. We can go back to our Django shell and test sending emails from the new and shiny domain

>>>send_mail(
...    'Subject here',
...    'Here is the message.',
...    'hello@yourdomain.com',
...    ['your.email.address+to.test@example.com']
...)
1

Here’s your email

Pretty neat, right? But it could be even more pretty…

Custom name in the email

Have you ever noticed a pretty email name? Something like this

It even includes a link to unsubscribe. You can be so cool too. This is very easy. You just need to modify the sender name. Add a desired name in the beginning in double quotes and an email address the email will be sent from in the end in <> like this ”Desired name” <email@example.com>.

>>>send_mail(
...    'Subject here',
...    'Here is the message.',
...    '"Hello, you" <hello@yourdomain.com>',
...    ['your.email.address+to.test@example.com']
...)
1

Now you have a cool name too

And it can be whatever you want it to.

Conclusion

Now you should know how to

set up django-ses library
use it to send emails
set up emails in SES
set up domains in SES
make email sender info prettier

This is the sixth part of a series of articles about the MuN. Stay tuned for part 7. You can find the code of this project, as well as my other projects, on my GitHub page. Leave your comments down below and follow me if you liked this article.

What you need to know to get started with AWS

Nikita Kholin — Thu, 07 Mar 2019 22:33:38 +0000

AWS is one of the most popular cloud computing services. AWS is short for Amazon Web Services and boy, they have lots of services.

Lots of application we use every day are hosted on AWS. They include Netflix, Airbnb, Slack to name a few.

Maybe you're working in a company that uses AWS and you want to know more about it. Or maybe you want to deploy your application to AWS. Or you’re just heard AWS somewhere and can’t get it out of your head. Anyways, here's what you'll need to know.

Servers

Your application, whether it’s your own or your it belongs to the company you work for, is probably running on a server. While you develop your application you're running it on your computer. But when you deploy the application you want to run it on someone else's computer.

AWS can offer that computer. It is called EC2 or Elastic Compute Cloud.

Also, if you’re into Data Science you can run your Jupyter notebooks there.

Database

Your application probably uses a database. PostgreSQL, MySQL, even Oracle, it doesn’t matter. What you need to know is that AWS has a service for databases and it's called RDS (Relational Database Service).

You could run your database on another EC2 instance like you’re running on your personal computer but RDS has lots of advantages. Here are a couple:

Cost-efficiency
Automatic software patching
Easy scaling
Automated backups
Monitoring

You can get familiar with the full list of features here.

Deployment

But how do you deploy your application? Well, there are many ways to do that on AWS. But if you're just getting started I would recommend using EB (Elastic Beanstalk).

It allows you to easily:

create environments with all configuration you need (EC2, RDS) and some that you maybe don’t even need at the start but it’s still good to have (load balancers, monitoring, notifications)
deploy to that environment
check logs and ssh to the server

EB supports deploying different platforms starting from Docker to Java to Ruby. You can check out the full list of supported platforms here.

If you've used Heroku in the past, EB is pretty similar to it. EB is pretty easy to set up compared to configuring each AWS service you need separately. But it is still not as easy as Heroku is.

If you would like to start deploying after you push a commit to git you can check out AWS CodeDeploy.

HTTP routing

Once your application is deployed you need to route your domain traffic to it. You can use Route53 to do that.

Although when you deploy with EB you can get a pretty decent domain like http://your-application.elasticbeanstalk.com/ it still has that feeling that it's not yours. You can buy a domain for your application directly on Route53. You can also delegate it to Route53 if you've bought it on another service like GoDaddy or NameCheap or whatever. You can use a dig console command to check whether your domain is migrated to AWS.

dig your-domain.com

Once you have your domain on AWS you need to add an A alias record to either

the domain EB gave you ([your-application.elasticbeanstalk.com](http://your-application.elasticbeanstalk.com/))
or your EC2’s Elastic IP
or your load balancer DNS name

If the term "A record" does not sound familiar to you I recommend reading this article about DNS.

Handling more traffic

At some point, your application will probably gain some traffic and your one EC2 instance will not be able to handle all the incoming requests. ELBs (Elastic Load Balancers) are already on AWS to rescue you.

Load balancers are routing incoming traffic to different servers. They are doing this using the “round-robin” algorithm. Round-robin algorithm is very simple — when a request comes in load balancer sends it to one server, a second request goes into a second server. And when all the servers received a request, load balancer starts from the beginning and sends a request to the first one again.

Making it secure (HTTPS)

If you want to add HTTPS to your website you would need to get a certificate. You can get certificates in ACM (AWS Certificate Manager). Once you get them you need to use them, they won’t automatically apply themselves. You can add the certificate to your EC2 instance. Or if you’ve deployed using EB you should’ve gotten a load balancer. In that case, you can apply the certificate to the load balancer and you now can serve requests via HTTPS.

However, in this simple case, people could still visit your application over HTTP. But you can force them to use HTTPS by redirecting them if they came from HTTP. Previously, you would need to set up NGINX or Apache redirection. But in some recent time, AWS released Application Load Balancers and they include an option to redirect using some custom rules. You can read how to set up HTTPS redirection using Application Load Balancers here.

Storing files

You often need to save files in your application. But storing them in the database may not be the most efficient solution. AWS has a service called S3 which stands for Simple Storage Service. It does what the name suggests — it simply stores data. You can write there anything you can think of. It's like a Google Drive or iCloud.

To store files on S3 you can use AWS SDK for your language. You can also look for other libraries because I’m sure there are more.

Sending emails

Another thing most applications do is sending emails. Emails have lost of uses: newsletters, system notifications just to name a few.

To send emails with AWS you need to use SES (Simple Email Service). You could use your personal email or register a new one for your application and send emails from those emails. And that's a pretty good starting point.

However, in the long run, sending emails from a personal email address is not a good choice. A better way would be to get a domain and send emails from it. To do that you need to verify your domain in SES. And after that, you can send emails using AWS SDK which supports multiple programming languages. Or you can find a library on your own or ever create one.

Conclusion

In this article, I've talked about the most popular AWS services and why and how they are used. Now you should know that you should

Run your servers on EC2
Run your database on RDS
Deploy to AWS using EB
Manage your domains in Route53
Get SSL certificates in ACM
Handle more traffic with ELB
Store files on S3
Send emails with SES

Of course, this list is just the tip of the AWS services iceberg but it should be enough to get you started.

If you liked this article be sure to leave your comments down below and follow me.

Telegram push notifications

Nikita Kholin — Thu, 14 Feb 2019 07:34:06 +0000

Telegram is a great platform with lots of great users (I'm a Telegram user myself). And what would be the best way for Telegram users to receive notifications? We can't know for sure. Maybe they like email or something else. But we can guess that sending notifications to Telegram would be is pretty convenient.

If you would like to send Telegram notifications from your application you've come to the right place. I've added this feature to my application and I love it.

One quick note. In this article I provide examples of code in Python. But the ideas are not Python-specific and can be translated into another language without any hustle.

So without further ado let's dive into how we can do it.

Create a Telegram bot

First of all, you need to create a Telegram bot. To do this you need to use another Telegram bot, BotFather. Just talk to him (press start).

Now you see what it can do. But what interests us is creating a new bot so that's what we're going to choose (/newbot).

You're quickly going to find out that bot's name should end with "bot". And since you're like me and coming to the game too late, most bot names are already taken.

But eventually, you're going to find a name for your bot and get an access token we're going to need.

Now that you have a bot, Telegram users can find and use it. But there is one problem — you can't associate users that come from Telegram to the users in your application. Let me show you why.

Once a user presses the "Start" button on your bot, you will receive an "update". You can check all bot's updates even in your browser. Visiting the following URL https://api.telegram.org/bot{bot_token}/getUpdates (don't forget to use your access token in the URL). Here's what I got

Can't read anything? Don't worry. You can fix it by installing some JSON prettifier extension in your browser. I use JSON Formatter for Chrome. It looks so much better.

So as you can see we don't get that much information about the person. From this information, we can get their full name. But it would be lucky if the user would provide their full name in your application and doesn't guarantee uniqueness of it. So we can't use that to find a user in your applications.

Another piece of information we get is the username. That is more useful as it is unique between all Telegram users. But most likely you don't have that available in your applications. So we would need to ask a user to enter their username somewhere in the application. It's just too much of work that I'm not sure anyone would do.

Another option to associate a user would be to ask them to write an email that they used in your application to the bot. But this has too many flaws: the user can make a typo while entering the email, the user can enter the email of another user and exploit the system. This is just too bad.

Can we do better?

Associating a user

Of course we can. To associate the user we'll use a technique called deep linking.

First, you need to create a random unique token for each user. I've used the following code to generate the token using Python

from secrets import token_urlsafe
token = token_urlsafe(8)
token
# => 'uEDbtJFHxKc'

Then you need to save that token to be able to find a user with it later. You can save it to your database or use some other place like a cache for example. I have a Notification model so I added a field to a model's table.

class Notification(models.Model):
    user = models.ForeignKey(User, on_delete=models.CASCADE)
    # ...
    connect_token = models.CharField(max_length=64, null=True)

So we've generated uEDbtJFHxKc token and saved it. Now we need to use this token in a URL to the telegram bot which the user needs to click to make it all work

telegram_url = 'https://www.telegram.me'
bot_name = 'music_notification_bot'
token = 'uEDbtJFHxKc'
url = f'{telegram_url}/{bot_name}?start={token}'

Now that we have our URL, 'https://telegram.me/music_notification_bot?start=uEDbtJFHxKc', it's time to show it to the user. Just display it in any place of your application and wait for the user to click on it.

Once the user takes the bait and clicks "Start" you should receive another update

{
    "ok": true,
    "result": [
        // ...
        // previous updates
        // ...
        {
            "update_id": 599162365,
            "message": {
                "message_id": 174,
                "from": { ... },
                "chat": { ... },
                "date": 1549788357,
                "text": "/start uEDbtJFHxKc",
                "entities": [ ... ]
            }
        }
    ]
}

We can finally identify our user. The text field now contains our user token. Let's go ahead and take it out of this field

bot_token = 'your_bot_token'
updates_url = f'https://api.telegram.org/bot{bot_token}/getUpdates'

import requests
response = requests.get(updates_url).json()
text = response['result'][0]['message']['text']
text
# => '/start uEDbtJFHxKc'

splitted_text = text.split(' ')
# => ['/start', 'uEDbtJFHxKc']

token = splitted_text[-1]
# => 'uEDbtJFHxKc'

This token can be used to find the user. Your implementation depends on the way you saved the token in the first place. But here's how I do it

notification = Notification.objects.get(channel='telegram', connect_token=token)
user = notification.user

So the user had pressed the "Start" button. But they see that nothing happened. Let's welcome them at least.

Checking for updates

To welcome the user we need to discover that the user had started a conversation with our bot. There are two options for how we can do that: polling and webhooks.

You already know what polling is. You've already done it. Or at least have seen me do it. Once we've checked out the https://api.telegram.org/bot{bot_token}/getUpdates page we did one poll. Polling is checking for updates constantly, every 2 seconds for example. This way we can always know when someone interacted with the bot.

Webhook takes a bit different direction. Instead of checking every 2 seconds for updates we just wait for an update to happens. And when it happens Telegram will send a request with the update data to a URL that we specify. This way we can give both our and Telegram servers some rest and just wait for the update to come.

Polling can be better if you've got high traffic but, unfortunately, it's rather an exception so I've decided to go with the webhook.

Webhooks

Setting a webhook in Telegram is very easy. You just need to send a request to https://api.telegram.org/bot{bot_token}/setWebhook?url={your_server_url}. Opening this link in your browser works too. your_server_url is the URL Telegram will send updates to. Here's what you should get in the response

{
    "ok": true,
    "result": true,
    "description": "Webhook was set"
}

If you don't trust yourself you can visit https://api.telegram.org/bot{bot_token}/getWebhookInfo just to doublecheck that everything is OK. You should see something like this

{
    "ok": true,
    "result": {
        "url": "https://example.com/your_server_endpoint",
        "has_custom_certificate": false,
        "pending_update_count": 0,
        "max_connections": 40
    }
}

Now. If something is not OK (like you've set a wrong URL) you can always delete webhook by visiting https://api.telegram.org/bot{bot_token}/deleteWebhook. And then setting the webhook again.

Local development

Before continuing I would like to say a few words about local development. Webhooks are not very suitable for it. Webhooks are sent to a URL and most likely you don't know your computer's URL. Also, Telegram webhook requires the URL to be secure (HTTPS).

But there is a solution to this problem — ngrok. ngrok is a tool that exposes your local environment to the world. Download ngrok, install it and start it with the port your server is running on. My server is running on 8000 port so I would need to run in a console

/path/to/ngrok http 8000

Then ngrok should give you a URL that you can use to set a webhook up.

Welcoming a user

Now that you're ready to develop, let's welcome our user, they're waiting for it.

Once the user clicks "Start" you Telegram will send an update to your server's URL. The interesting parts of the update should look like

{
    "message": {
        "chat": {
            "id": 457
        },
        "text": "/start uEDbtJFHxKc",
    }
}

This is a perfect time to associate user using message text. There is also an interesting piece of information, chat ID. Chat ID is what we need to send a message to that user. Telegram has an API endpoint to send a message that looks like this https://api.telegram.org/bot{bot_token}/sendMessage?chat_id={chat_id}&text={text}. I'm not sure I need to explain how to use it but here's what my code that processes the webhook looks like

import json
import requests

def callback(request):
    body = json.loads(request.body)
    text = body['message']['text']
    token = text.split(' ')[-1]
    associate_user_by_token(token)

    bot_key = os.environ.get('TELEGRAM_API_KEY')
    chat_id = body['message']['chat']['id']
    text = "Welcome!"

    send_message_url = f'https://api.telegram.org/bot{bot_key}/sendMessage?chat_id={chat_id}&text={text}'
    requests.post(send_message_url)

If we send a welcome message after the user clicked the famous "Start" button the user won't have any second thoughts whether everything is working or not.

Pushing notifications

Finally, we got to the point why we're doing all of this — to push notifications. You may want to notify the user about some information that happened in your application. For example, someone liked the user's post or whatever. I use Telegram to notify about new music releases from the user's favorite artists.

You already know hot to send notifications. You just need to send a message using https://api.telegram.org/bot{bot_token}/sendMessage?chat_id={chat_id}&text={notification_text}.

Of course, if you're planning to send notifications not only when the user interacts with the bot you need to save chat_id in your database.

You might also want to include links or some other formatting in your message. In this case, you would need to add another parameter to the send message URL, parse_mode. There are 2 parsing options: Markdown or HTML. I use Markdown since I find it simpler to use. If you're not familiar with Markdown you can use HMTL but I would recommend reading how easy Markdown is.

Here's how the send message URL looks with parse_mode parameter https://api.telegram.org/bot{bot_token}/sendMessage?chat_id={chat_id}&text={notification_text}&parse_mode=markdown.

I add links to new releases to the notifications text like so {release.date}: {release.artist.name} - [{release.title}]({release.url}). You can read more about how to format you messages here.

Also, there are more parameters available for the send message URL like disable_notification. There is always a place to explore.

Conclusion

Now you should know how to

create a bot in Telegram using BotFather
check for updates and what is better — webhooks or polling
associate user using deep linking
send a welcome message and continue sending notifications
format messages you send

Hopefully, this article was useful to you. This is the fifth part of a series of articles about the MuN. Stay tuned for part 6. You can find the code of this project, as well as my other projects, on my GitHub page. Leave your comments down below and follow me if you liked this article.

Explaining how OAuth works with Spotify as an example

Nikita Kholin — Thu, 24 Jan 2019 08:07:05 +0000

Have you ever wondered what is OAuth, how it works, and why any more or less popular website implements it? In this article, we'll explore those questions and will write code in Python to use OAuth with Spotify.

What is it?

According to Wikipedia, OAuth is

OAuth is an open standard for access delegation, commonly used as a way for Internet users to grant websites or applications access to their information on other websites but without giving them the passwords.

Sounds pretty logical. Let's use a Spotify to make it more specific.

OAuth is commonly used as a way for Internet users to grant websites or applications (your website or application) access to their information (like their favorite artists, or ability to add a new artist to favorites) on other websites (Spotify) but without giving them the passwords.

One more thing. "OAuth is an open standard" which means that it's going to work about the same for all other websites (like Facebook, Twitter) that are implementing OAuth.

How it works

On the high level, you (are also called client, and the website you are trying to OAuth with is called provider), get a user's access token (which is essentially a password) and with it, you can use all the accesses that you've listed in a so-called "scope".

A user clicks a link to the provider website that will request authorization
The user sees an authorization screen

The user accepts your scope
The user is redirected to a "redirect URL" with an access token
Now you can make requests for the data you need with the access token

But what do we need to make it work?

Create a "Client Application"

First of all, you need to create a "Client Application". Its main purpose is to know where to redirect the user on step #4.

Another purpose of the Client Application is to show the name and a brief description of your project on the authorization screen on step #2.

You can google find a place where to do this with a query something like "#{PROVIDER_NAME} oauth app".

Build a link to the "Provider"

First, you need to find how to build a link. Google can help us again. Just google "#{PROVIDER_NAME} oauth URL". You should find something like official documentation from the provider.

Once we've found how to build the link you would most likely need to use a couple parameters in it:

client_id — an identifier of your client application so that the provider knows where to redirect the user
redirect_uri — the famous redirect URL (the provider could use the one specified in the Client Application but usually you can enter multiple URLs there and this parameter is for the clarification)
scope — a list of accesses that your project needs (you can find scopes in the official documentation).

Please refer to the official documentation for exact names of the parameters listed above.

I will show what I would have to build in my case. Regarding the scope, I need user-read-email and user-follow-read to read user's email and what they follow. To start I would point to the URL where the server is running

provider_url = "https://accounts.spotify.com/authorize"

from urllib.parse import urlencode
params = urlencode({
    'client_id': 'MY_CLIENT_ID',
    'scope': ['user-read-email', 'user-follow-read'],
    'redirect_uri': 'http://127.0.0.1:5000/spotify/callback',
    'response_type': 'code'
})

url = provider_url + '?' + params
url
# => 'https://accounts.spotify.com/authorize?client_id=MY_CLIENT_ID&scope=%5B%27user-read-email%27%2C+%27user-follow-read%27%5D&redirect_uri=http%3A%2F%2F127.0.0.1%3A5000%2Fspotify%2Fcallback&response_type=code'

Create an endpoint for the redirect URL

On the step #4 user is redirected to the redirect URL. This URL is where an access token with user's info will come to.

When I've been building my link to the provider, I've used 'http://127.0.0.1:5000/spotify/callback' as a redirect URL. This link needs to point to an endpoint on whatever server you have.

Here's a quick example how to set up an endpoint in Flask

from flask import Flask
app = Flask(__name__)

@app.route("/spotify/callback")
def spotify_callback():
    return "You finally called me back!"

In the callback you've just created, you can do whatever you want with the user's data. You can create a user's profile on your project, send him an email or whatever. But if you want to get more data from this user later it's better to save a token that comes with the request to your database.

Testing whether it's working

Now that you got your access token, you can do the things that you've requested in the scope.

I've requested user-follow-read so now I can read what artists the user is following to notify when a new album comes out. Spotify has an endpoint to fetch user's followed artists so let's just use it

artists = []
all_artists_loaded = False
limit = 50
access_token = fetch_access_token() # your function to pull the token
                                    # from the place where you saved it
url = f"https://api.spotify.com/v1/me/following?type=artist&limit={limit}&access_token={access_token}"

while not all_artists_loaded:
    response = requests.get(url).json()['artists']
    current_request_artists = response['items']
    artists += current_request_artists
    if response['next']:
        url = response['next'] + f"&access_token={access_token}"
    else:
        all_artists_loaded = True

print(artists)

If you'd like more info on how to fetch latest albums you can read my previous article.

Save the refresh token too

Sadly, access token usually expires. But luckily we can request a new one with the refresh token that comes with the request token.

Once the access token is expired you need to send a request with the refresh token to the endpoint that is specified in the docs. We can do that with requests library in Python

refresh_token = pull_refresh_token() # your function to pull the token
                                     # from the place where you saved it
client_id = "YOUR_CLIENT_ID" # that you received once you created
                             # the Client Application
client_secret = "YOUR_CLIENT_SECRET" # that you received once you created
                                     # the Client Application too

refresh_url = "https://accounts.spotify.com/api/token" # from the docs

payload = {
    'refresh_token': refresh_token,
    'grant_type': 'refresh_token'
}

auth_header = base64.b64encode(six.text_type(client_id + ':' + client_secret).encode('ascii'))
headers = {'Authorization': 'Basic %s' % auth_header.decode('ascii')}

response = requests.post(refresh_url, data=payload, headers=headers)

token_info = response.json()

Source of the code above.

The URL (as well as possible other required parameters) have to be, once again, found in the official documentation.

Why everyone implements OAuth?

It's just convenient for the end-user. OAuth takes the burden of creating passwords or fake accounts from users and places it on developers. But now it shouldn't be too hard for you to use any provider and leverage the full power of users' data.

Don't invent a bicycle

There are always existing solutions to almost any problem you face. Always make sure to google before inventing the bicycle. If you need authentication you, there are already libraries for that. If you need to access some API, like a needed to use Spotify's, there are libraries for that too.

If you'd like to read about OAuth more in-depth there is an official guide that covers probably everything written in a very easy to read form.

Also, this is the fourth part of a series of articles about the MuN. Stay tuned for part 5. You can find the code of this project, as well as my other projects, on my GitHub page. Leave your comments down below and follow me if you liked this article.

Getting your latest releases from Deezer with Python

Nikita Kholin — Thu, 13 Dec 2018 22:51:18 +0000

Have you ever felt like you've missed an album from your favorite artist? If yes then this article is just for you.

Here we will fetch your followed artists and their albums from Deezer API to see whether you missed any releases or not. We will use Python to do all the work.

Getting started

First, you'll need to know your ID. You can get it by opening Deezer in a browser and visiting your profile.

Now, if you check the URL address you should see something like this

https://www.deezer.com/en/profile/700513741

A number at the end of the URL is your ID (mine is 700513741). Let's store it in a variable.

deezer_id = 700513741

If you want to get IDs of users of your project you'll need to create a Deezer application and connect that application to your project. For more information on how to do that you can check out this guide.

Fetching information about you

To begin our fetching journey let's fetch ourselves first. To fetch a user we can use Deezer's User endpoint. We just need to send a request to a URL specified in the documentation and information about a user should be in the response.

But before doing any requests with Python we can try doing some requests in Deezer's API explorer.

The docs say that the URL for getting info about a user is https://api.deezer.com/user/me. but if we try to perform a request to the endpoint we receive an error

An active access token must be used to query information about the current user

This means that we have to get a token that is related to our user for Deezer to know who is "me". But we all like doing things in an easy way and getting that access token is just too hard. This is exactly why I asked you to get your ID by yourself. Now we can just replace /me with your ID and everything should be working. Here's what I got in the API explorer

Making real requests

Making requests in Python is very easy with requests library. If you don't have the library installed you can use an official installation guide. Once you have requests installed we can proceed to coding.

To send a regular GET request with requests the library provides us a get method which just needs a URL

import requests
url = f"https://api.deezer.com/user/{deezer_id}"
user_response = requests.get(url)

The user_response variable should now contain lots of data that happened during the request. What interests us is what the body of the response. We can get it with response.content method

user_response.content
# => b'{"id":700513741, "name":"hmlON", ...}'

As we can the response contains a JSON string. We could parse it with a json library but requests have worked everything out for is — we can just use a json() method

user_response.json()
# => {'id': 700513741, 'name': 'hmlON', ...}

Now we have a normal dictionary with which we can work. Let's finish our "hello world" program (Deezer API edition) before we continue making other requests

name = user_response.json()['name']
print(f"Hello, {name}!")

Fetching your favorite artists

Your favorite artists can be found on a User / Artists endpoint. As usual, it suggests to use /me in the URL but we're smarter then that and will use the ID. So let's test the endpoint.

artists_url = f"https://api.deezer.com/user/{deezer_id}/artists"
artists_response = requests.get(artists_url)
artists_response.json()
# => {
#  "data": [
#    {
#      "id": "296861",
#      "name": "The Neighbourhood",
#      ...
#    },
#    ...
#  ]
#  ...
#  "total": 30,
#  "next": "https://api.deezer.com/user/700513741/artists?index=25"
# }

The endpoint seems to be working. As we can see from the response, we have a couple of interesting fields here:

data — your favorite artists that we wanted
total — how many artists we are following
next — a link to the next page of results

The next parameter is empty if a response returned the last page of data. This means that if we want to just take a glance at artists we can send requests as is. But if we want to fetch all data (like we do now) we would have to use this parameter.

The algorithm for fetching all records is pretty simple — we go to the next page while it is present. Let's go ahead and implement it

artists = []               # we'll store artists here as we go through the pages
all_artists_loaded = False # in the beginning nothing is loaded
url = f"https://api.deezer.com/user/{deezer_id}/artists"

while not all_artists_loaded:
    response = requests.get(url).json()
    artists += response['data']   # data attribute contains artists from the request
    if response.get('next'):      # there are more artists on the next page
        url = response['next']    # we will continue to fetch from the next page
    else:                         # there are no more followed artists
        all_artists_loaded = True # we will end the loop

The algorithm is pretty straightforward but there is one thing I would like to note. When I'm checking whether there is a next page I use response.get('next') instead of response['next']. The difference here is that when the next key is not present in the response the response.get('next') method will return nothing while the regular response['next'] will raise an error.

response = {'data': ['...']} # response without the 'next' attribute

response['next']
# raises KeyError: 'next'

response.get('next')
# => None

Fetching your favorite artists' releases

Now that we have all followed artists we can fetch their albums. The Artist / Albums endpoint is quite suitable for this task.

This time we would need to loop over each artist and save all albums in an array. To fetch all releases for each artist we would use the same algorithm as we've used to fetch artists.

releases = [] # we'll store all releases here

for artist in artists:
    # the old algorithm is still working here
    all_releases_loaded = False
    artist_id = artist['id']
    url = f"https://api.deezer.com/artist/{artist_id}/albums"

    while not all_releases_loaded:
        response = requests.get(url).json()
        releases += response['data']
        if response.get('next'):
            url = response['next']
        else:
            all_releases_loaded = True

Pretty much the same as before. But there is one problem which we will notice if we look at any release

releases[-1]
# {
#  'cover': 'https://api.deezer.com/album/40571541/image',
#  'explicit_lyrics': True,
#  'fans': 52,
#  'genre_id': 116,
#  'id': 40571541,
#  'link': 'https://www.deezer.com/album/40571541',
#  'record_type': 'single',
#  'release_date': '2017-04-18',
#  'title': 'DEADMAN WUNDERLAND',
#  'tracklist': 'https://api.deezer.com/album/40571541/tracks',
#  'type': 'album'
# }

The problem is that we don't know who the artist is. We can solve it in two ways:

by storing artists in a dictionary with their ID as a key and adding artist_id to all releases
by storing full title (with artist's name) in the release

I will go with the second one since it would be easier to do but if you would like to implement the first solution or any other please share it in the comments.

To save the full title in a release we would need to store artist's songs in a temporary array. And when all albums are fetched we would need to add the full title to them.

releases = [] # we'll store all releases here

for artist in artists:
    # this time we'll store releases in a temporary array
    current_artist_releases = []

    # nothing changed here
    all_releases_loaded = False
    artist_id = artist['id']
    url = f"https://api.deezer.com/artist/{artist_id}/albums"

    while not all_releases_loaded:
        response = requests.get(url).json()

        # now we're saving each page's response to current_artists_releases
        # instead of straight to releases
        current_artist_releases += response['data']

        # this stays the same too
        if response.get('next'):
            url = response['next']
        else:
            all_releases_loaded = True

    # and after we've fetched all releases
    # we can add full title to all of them
    for release in current_artist_releases:
        full_title = f"{artist['name']} — {release['title']}"
        release['full_title'] = full_title

    # and only after we every release from current artist
    # has a full title we can add them to the main releases
    releases += current_artist_releases

So now all releases should have a full title

releases[-1]['full_title']
# => 'Aries — DEADMAN WUNDERLAND'

So what have you missed?

Let's check what was released during the last year. If you haven't checked it, you might be surprised by some interesting albums.

To begin, we need to filter releases so that only that were created during the last year are left. To do this we will use a standard datetime library to get a date of the last year

import datetime

today = datetime.date.today()
print(today)
# => 2018-12-13

year_ago = today - datetime.timedelta(days=365)
print(year_ago)
# => 2017-12-13

Then, to filter we need to compare the release date with our year_ago variable. The only problem here is that release date is a string

releases[-1]['release_date']
# => '2017-04-18'

And we can't compare string with a date in Python. So a way out of this is to convert a string into a date. The datetime library has a function just for our case called datetime.strptime. It needs a string (the release date) and it's format.

As we can see, release_date format is 'year-month-day'. With a little help of documentation, this converts into '%Y-%m-%d'

datetime.datetime.strptime(releases[-1]['release_date'], '%Y-%m-%d').date()
# => datetime.date(2017, 4, 18)

Let's store the release_date as a date instead of a string for all releases.

for release in releases:
    previous_date = release['release_date']
    new_date = datetime.datetime.strptime(previous_date, '%Y-%m-%d').date()
    release['release_date'] = new_date

Now we can finally filter our releases to get only the ones from last year

latest_releases = [
    release for release in releases
    if release['release_date'] > year_ago
]

After that, all of the last year's releases are stored in the latest_releases variable. Now we need to sort those releases by date to have them listed chronologically

latest_releases = sorted(latest_releases, key=lambda release: release['release_date'])

And finally, we can print those releases

for release in latest_releases:
    date = release['release_date']
    title = release['full_title']
    print(f"{date}: {title}")
# ...
# 2018-10-25: Aries — BLOSSOM
# 2018-11-02: Panic! At the Disco — The Greatest Show
# 2018-11-02: The Neighbourhood — Hard To Imagine The Neighbourhood Ever Changing
# 2018-11-09: Muse — Simulation Theory (Super Deluxe)
# 2018-11-09: Imagine Dragons — Origins
# 2018-11-09: Imagine Dragons — Origins (Deluxe)
# 2018-11-09: Goody Grace — Nostalgia Is A Lie
# 2018-11-30: Arctic Monkeys — Tranquility Base Hotel & Casino

If you've followed along this far please share your latest releases for the last month in the comments.

Conclusion

In this article I've covered:

how to work with Deezer API
how to make requests with Python
how to convert strings into dates

Congratulations, now you know how to check for new releases from artists that you follow. If you're like me and hate missing out on new releases from your favorite artists, I've built a website just for this purpose.

If you would like to run the code by yourself and don't want to scramble it from the article pieces by pieces, I've uploaded it into a gist on GitHub.

Also, this is the third part of a series of articles about the MuN. Stay tuned for part 4. You can find the code of this project, as well as my other projects, on my GitHub page. Leave your comments down below and follow me if you liked this article.

Django authentication via Google, Deezer, and Spotify

Nikita Kholin — Thu, 22 Nov 2018 19:38:18 +0000

In this tutorial we will add an ability to sign up, sign in via Google, Deezer, and Spotify, view a success screen, and sign out afterward. Our project is going to be about music releases notifications.

I've already created a Django project with a landing page in a previous tutorial.

How will we do that?

We're going to use a social-app-django library. It supports a huge number of auth providers. They are also called "Backends" in the library. As you could guess, we will need Google, Deezer, and Spotify.

To install the library please follow an official installation guide.

Adding social-app-django to your Django project

First of all, you need to add the library to our INSTALLED_APPS in your project's settings.py

INSTALLED_APPS = [
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    # ...
    'social_django',
    # ...
    'pages',
]

After that, migrate your database by running

python3 manage.py migrate

Then add social_django.urls to your project's urls.py

urlpatterns = [
    # ...
    path('social/', include('social_django.urls')),
    # ...
]

And SOCIAL_AUTH_URL_NAMESPACE to your project's settings.py again

SOCIAL_AUTH_URL_NAMESPACE = 'social'

Now you should have nothing, but we're ready to add an auth provider to the project. For more information, you can read the official python-social-auth installation guide for Django.

Authentication via Google(+)

First, you need to get credentials for the application

Visit Google API Console
Go to credentials
Create credentials
Add http://127.0.0.1:8000/social/complete/google-plus/ to Authorized redirect URIs
Here you'll have to add your production redirect URL once you have one

Now you should see Client ID and Client secret in your app. Add them to project's settings.py

SOCIAL_AUTH_GOOGLE_PLUS_KEY = 'Your Client ID'
SOCIAL_AUTH_GOOGLE_PLUS_SECRET = 'Your Client secret'

Add Google to AUTHENTICATION_BACKENDS in project's settings.py

AUTHENTICATION_BACKENDS = (
    'social_core.backends.google.GooglePlusAuth',
)

And finally, add a link to sign in via Google to your menu or header or wherever you want your users to click to sign in

<a href="{% url "social:begin" "google-plus" %}">Google</a>

For more reading about how authentication via Google works here's the official guide.

Authentication via Deezer

The process for Deezer authentication is pretty similar to Google's one. First, getting credentials

Visit Deezer for developers
Create an app
Set http://127.0.0.1:8000/social/complete/deezer/ to Redirect URL after authentication
And again, here you'll have to add your production redirect URL once you have one

Now you should see Application id and a Secret Key in your app. Add them to project's settings.py

SOCIAL_AUTH_DEEZER_KEY = 'Your Application id'
SOCIAL_AUTH_DEEZER_SECRET = 'Your Secret Key'

Again, add Deezer to AUTHENTICATION_BACKENDS in project's settings.py

AUTHENTICATION_BACKENDS = (
    'social_core.backends.google.GooglePlusAuth',
    'social_core.backends.deezer.DeezerOAuth2',
)

And again, a link to sign in via Deezer

<a href="{% url "social:begin" "deezer" %}">Deezer</a>

One more thing

You might want to ask for some permissions. For example, Deezer does not provide user's email by default. You need to ask a permission for email. You can check out a full list of available permissions in official permissions documentation.

Adding required permissions is pretty simple with the social-app-django library. You just need to add permission you want to a scope constant in the project's settings.py. The constant, in Deezer case, has to be called SOCIAL_AUTH_DEEZER_SCOPE

SOCIAL_AUTH_DEEZER_SCOPE = ['basic_access', 'email']

Authentication via Spotify

Spotify authentication is as similar to Google as Deezer is.

Visit Spotify for developers
Create a Client ID
Set http://127.0.0.1:8000/social/complete/spotify/ to Redirect URIs
And again, here you'll have to add your production redirect URL once you have one

Now you should see Client ID and a Client Secret in your app. Add them to project's settings.py

SOCIAL_AUTH_SPOTIFY_KEY = 'Your Client ID'
SOCIAL_AUTH_SPOTIFY_SECRET = 'Your Client Secret'

Again, add Spotify to AUTHENTICATION_BACKENDS in project's settings.py

AUTHENTICATION_BACKENDS = (
    'social_core.backends.google.GooglePlusAuth',
    'social_core.backends.deezer.DeezerOAuth2',
    'social_core.backends.spotify.SpotifyOAuth2',
)

And again, a link to sign in via Spotify

<a href="{% url "social:begin" "spotify" %}">Spotify</a>

Regarding permissions, I need user-read-email and user-library-read. You can check out a full list of available permissions in official permissions documentation.

SOCIAL_AUTH_SPOTIFY_SCOPE = ['user-read-email', 'user-library-read']

Life after sign in

Usually, after a sign in it's a good idea to show some sort of dashboard or feed or whatever is the main purpose of your application. I'm going to show a screen with all the latest music releases.

First, let's create a releases app

python3 manage.py startapp releases

Add the app to installed apps in the project's settings.py

INSTALLED_APPS = [
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'social_django',
    'pages',
    'releases',
]

Add URLs to the project's URLs in the project's urls.py

urlpatterns = [
    # ...
    path('', include('releases.urls')),
    # ...
]

Define a /releases URL in releases/urls.py

from django.urls import path
from . import views

urlpatterns = [
    path('releases', views.index, name='index'),
]

Create a view in releases/views.py

def index(request):
    return render(request, 'releases/index.html')

Create a simple template to show that we care about our users in releases/templates/releases/index.html (I hate to have to write releases two times too)

<h1>Latest Releases</h1>
<p>We care about you, but we don't have anything done yet.</p>

Now that we have somewhere to redirect a user after a sign in, there is only one thing left to do — set a LOGIN_REDIRECT_URL in the project's settings.py

LOGIN_REDIRECT_URL = '/releases'

After this line is added all users should go directly to /releases URL after they sign in.

Fixing duplicate users

Now that we have the ability to sign in with a couple different accounts there is one problem — we don't do anything with duplicate users, users that have identical email. We can quickly check that by signing up 2 times via two different services (with Google and Spotify for example).

Then we can check that there are 2 users by running a Django shell

python3 manage.py shell

And checking current count of users there

from django.contrib.auth.models import User
User.objects.count()

We can also investigate what is the difference between them

vars(User.objects.all())

To fix this problem we'll have to learn a bit more how social-app-django works.

When a user signs in they go through a so-called pipeline. Here's how the default pipeline looks like

(
    # Get the information we can about the user and return it in a simple
    # format to create the user instance later. On some cases the details are
    # already part of the auth response from the provider, but sometimes this
    # could hit a provider API.
    'social_core.pipeline.social_auth.social_details',

    # Get the social uid from whichever service we're authing thru. The uid is
    # the unique identifier of the given user in the provider.
    'social_core.pipeline.social_auth.social_uid',

    # Verifies that the current auth process is valid within the current
    # project, this is where emails and domains whitelists are applied (if
    # defined).
    'social_core.pipeline.social_auth.auth_allowed',

    # Checks if the current social-account is already associated in the site.
    'social_core.pipeline.social_auth.social_user',

    # Make up a username for this person, appends a random string at the end if
    # there's any collision.
    'social_core.pipeline.user.get_username',

    # Send a validation email to the user to verify its email address.
    # Disabled by default.
    # 'social_core.pipeline.mail.mail_validation',

    # Associates the current social details with another user account with
    # a similar email address. Disabled by default.
    # 'social_core.pipeline.social_auth.associate_by_email',

    # Create a user account if we haven't found one yet.
    'social_core.pipeline.user.create_user',

    # Create the record that associates the social account with the user.
    'social_core.pipeline.social_auth.associate_user',

    # Populate the extra_data field in the social record with the values
    # specified by settings (and the default ones like access_token, etc).
    'social_core.pipeline.social_auth.load_extra_data',

    # Update the user record with any changed info from the auth service.
    'social_core.pipeline.user.user_details',
)

If you like to, you can read a description for each stage of the pipeline. But what interests us at the moment is the 7th step, social_core.pipeline.social_auth.associate_by_email which is disabled by default. What it does is it

Associates the current social details with another user account with a similar email address

Sounds like exactly what we need. Just uncomment the line with this setting and copy the pipeline to your project's settings.py

SOCIAL_AUTH_PIPELINE = (
    'social_core.pipeline.social_auth.social_details',
    'social_core.pipeline.social_auth.social_uid',
    'social_core.pipeline.social_auth.auth_allowed',
    'social_core.pipeline.social_auth.social_user',
    'social_core.pipeline.social_auth.associate_by_email',
    'social_core.pipeline.user.create_user',
    'social_core.pipeline.social_auth.associate_user',
    'social_core.pipeline.social_auth.load_extra_data',
    'social_core.pipeline.user.user_details',
)

You can read more about in the official docs for pipelines. We'll build a custom pipeline in the next part of the series.

Signing out

You might want to add an ability for your users to sign out. I would not recommend doing that since this is just another opportunity for a user to leave your site but In case you still want to add it, it's pretty easy.

Django has an already built-in support for authentication called django.contrib.auth. Right now, we'll need just the sign-out part of it.

First, add django.contrib.auth.urls to your project's urls.py

urlpatterns = [
    # ...
    path('', include('django.contrib.auth.urls')),
    # ...
]

Then, set LOGOUT_REDIRECT_URL to '/' in your project's settings.py so that users are redirected to the homepage after they sign out

LOGOUT_REDIRECT_URL = '/'

Now, all that is left is to add a sign-out link wherever you want

<a href="{% url 'logout' %}" class="header-link">Log out</a>

Showing only relevant links

But right now your user can see all sign in links as well as a sign-out link. You probably wouldn't want a user to sign in or out 2 times and what does that even mean? So let's show only relevant links.

Django's user model has a special attribute for that called is_authenticated that we can use. We can just use a simple if in our template and check if a user, a variable which is already available in our templates, is_authenticated.

{% if user.is_authenticated %}
    <a href="/logout">Log out</a>
{% else %}
    <a href="{% url "social:begin" "google-plus" %}">Google</a>
    <a href="{% url "social:begin" "deezer" %}">Deezer</a>
    <a href="{% url "social:begin" "spotify" %}">Spotify</a>
{% endif %}

Conclusion

In this story I’ve covered:

how to use a social-app-django library
how to create apps, set custom scopes in Google, Deezer, and Spotify and of course integrate those services into your application
how to create a simple success page to show after the sign in
what is a pipeline in social-app-django library
and how to sign users out

This is the second part of the series of articles about the MuN. Stay tuned for part 3. You can find the code of this project, as well as my other projects, on my GitHub page. Leave your comments down below and follow me if you liked this article.

Creating a landing page in Django

Nikita Kholin — Mon, 12 Nov 2018 22:20:28 +0000

In this tutorial we will create a landing page for a music releases notifications application in Django. To begin, we need to create a Django project.

Creating a Django project

I will assume you have Django installed but in case you don't, you can use the official installation guide.

To create a project run the following command in your terminal

django-admin startproject mun

And after the project is created go to its directory via cd command

cd /mun

And starting a server is as simple as

python manage.py runserver

After visiting 127.0.0.1:8000 you should see a screen that everything is great...

Except it's not your landing page...

Where do we start?

To add a landing page we need to create an "app" which is a Django concept for things that are not coupled to your app and can be extracted from it (for example authentication which can be used in other applications too).

But let's not get into that too much and just create a simple app called "pages" which will only contain static pages like "landing", "contact us" and "about".

python manage.py startapp pages

When the app is ready, we need to add it to Django's INSTALLED_APPS. We'll Just append our app to the end of those other ones because we're going to need some of those.

INSTALLED_APPS = [
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'pages',
]

The newly created pages app has a new pages folder with a couple of python files. Particular ones that interest us are pages/urls.py and pages/views.py.

URLs

When a request comes to the server, Django has to find a "view", a function that will process the request and send the response. The way Django finds how to connect a URL that came with the request and a view is via urlpatterns which are located in your project's main directory. In my case, it's mun/urls.py.

In order to show our landing page on 127.0.0.1:8000 and essentially on our main domain we need to add the following code to pages/urls.py

from django.urls import path
from . import views

urlpatterns = [
    path('', views.index, name='index'),
]

A path is what comes after a domain in a URL. For example, in the URL https://www.google.com/search?q=mun the path would be /search?q=mun. Since we want our landing page to be on the / path the first parameter we provide to the path method is ''. The second argument is what view we need to render which is an index view in our case.

We also need to add urlpatterns from our app to the main project's urlpatterns with a built-in function include

from django.contrib import admin
from django.urls import include, path

urlpatterns = [
    path('', include('pages.urls')),
    path('admin/', admin.site.urls),
]

The view

If you recall, the index view is not yet created so let's go ahead and create it in pages/views.py

from django.shortcuts import render

def index(request):
    return render(request, 'pages/index.html')

Our index action simply renders a 'pages/index.html' template which is again, not yet created. So let's create it.

The template

I want the landing page to contain the project's name, a brief description and a list of features.

First, let's start with the header. Our sign in buttons won't do anything right now so their href is # (which means it won't do anything) for now. We'll fix that in the next story.

<header class="header">
  <div class="header-text">Sign in via:</div>
  <a href="#" class="header-link">Spotify</a>
  <a href="#" class="header-link">Deezer</a>
</header>

Second, let's add the brand name and a description

<section class="landing-section landing-main">
  <h1 class="brand">MuN</h1>
  <h2>
    <span class="brand-text">Mu</span>sic
    <span class="brand-text">N</span>otifications
  </h2>
  <p class="landing-main-text">
    Got tired of missing out on releases from your favourite artists?
    <br>
    You have found a perfect solution...
  </p>
</section>

Third, a list of features:

<section class="landing-section langing-features">
  <section class="landing-feature feature-music-services">
    <h2>Connect your favourite <span class="brand-text">mu</span>sic services</h2>
    <ul>
      <li>Spotify</li>
      <li>Deezer</li>
    </ul>
  </section>
  <section class="landing-feature feature-notifications">
    <h2>Get <span class="brand-text">n</span>otifications wherever you want them</h2>
    <ul>
      <li>Email</li>
      <li>Telegram</li>
    </ul>
  </section>
</section>

However, if we check our browser now we'll see a not so pleasant picture:

A CSS is a static file

To add CSS to the template Django has a whole app called django.contrib.staticfiles. By default , Django looks for static files in static folders of all apps. To add styles to the template we need to tell Django that we're going to use static files and add a stylesheet tag like so:

{% load static %}
<link rel="stylesheet" type="text/css" href="{% static 'pages/style.css' %}">

We'll also need the following line so that our page looks good on mobile

<meta name="viewport" content="width=device-width, initial-scale=1">

And while we're at it, let's add a title for our page

<title>MuN: Music Notifications</title>

Here's the full code glued together:

{% load static %}
<title>MuN: Music Notifications</title>
<meta name="viewport" content="width=device-width, initial-scale=1">
<link rel="stylesheet" type="text/css" href="{% static 'pages/style.css' %}">

<header class="header">
  <div class="header-text">Sign in via:</div>
  <a href="#" class="header-link">Spotify</a>
  <a href="#" class="header-link">Deezer</a>
</header>
<section class="landing-section landing-main">
  <div class="landing-section-content">
    <h1 class="brand">MuN</h1>
    <h2>
      <span class="brand-text">Mu</span>sic
      <span class="brand-text">N</span>otifications
    </h2>
    <p class="landing-main-text">
      Got tired of missing out on releases from your favourite artists?
      <br>
      You have found a perfect solution...
    </p>
  </div>
</section>
<section class="landing-section langing-features">
  <section class="landing-feature feature-music-services">
    <h2>Connect your favourite <span class="brand-text">mu</span>sic services</h2>
    <ul>
      <li>Spotify</li>
      <li>Deezer</li>
    </ul>
  </section>
  <section class="landing-feature feature-notifications">
    <h2>Get <span class="brand-text">n</span>otifications wherever you want them</h2>
    <ul>
      <li>Email</li>
      <li>Telegram</li>
    </ul>
  </section>
</section>

Adding some style

To begin with, I would like to have a full-screen picture in the background with the project's name and the description in the middle.

We can do the text in the middle with the flexbox. First, we need to make our landing-section container width and height to full screen. CSS has a special unit for that: viewport height (or vh) and viewport width (or vw). If we set them both to 100 it would take the whole screen (or viewport).

However, if we have too much content and the screen is too small (a mobile phone for example) then we'd need to show more content than 100vh. We can use min-height instead of height for that.

.landing-section {
  min-height: 100vh;
  width: 100vw;
}

Now that we have our container take up the full screen we can position the content in the center with the flexbox. First of all, we need to set display: flex to use flexbox. Then we can use align-items: center and justify-content: center to position the content in the center.

There is only one problem left which is the direction of the content. The fix for it is pretty strait-forward — adding flex-direction: column.

For more flexbox, I recommend reading this guide.

Here's how our final landing-section class looks like

.landing-section {
  min-height: 100vh;
  width: 100vw;
  display: flex;
  align-items: center;
  justify-content: center;
  flex-direction: column;
}

The image in the background

I've used Unsplash to find the image that would fit my preference.

I set the image as a background with a little help of CSS-Tricks.

.landing-main { 
  background: url(https://images.unsplash.com/photo-1460667262436-cf19894f4774) no-repeat center center fixed; 
  background-size: cover;
}

However, the image has light and dark colors and I couldn't use neither black nor white colors over it. So I decided to add a little overlay and make the text white (or rather light-grey because white is too strong).

.landing-section-content {
  padding: 2rem;
  background: rgba(0,0,0,0.5);
  width: 100vw;
  display: flex;
  align-items: center;
  justify-content: center;
  flex-direction: column;
  color: #ccc;
}

.brand {
  margin: 0;
  color: #c55e00;
}

.brand-text {
  color: #c55e00;
}

And a bit of styling for the header...

.header {
  width: 100vw;
  display: flex;
  justify-content: flex-end;
  top: 0;
  left: 0;
  position: absolute;

  background: rgba(0,0,0,0.5);
}

.header-link {
  margin: 1rem;
  border-bottom: 3px solid #c55e00;
  font-size: 1.2rem;
  color: #222;
  text-decoration: none;
  color: #ccc;
}

.header-text {
  margin: 1rem;
  font-size: 1.2rem;
  color: #ccc;
}

And here's what I got:

If you have any ideas for the features part, please, write them in the comments because it currently looks pretty awful.

One more thing

We've got most things done by now, however, there is one more. It's the icon. I like LogoMark website for this purpose. It is very simple to use and has a lot of beautiful icons and stuff.

When you download the icon, it gets saved in .png extension but .ico is better for an icon. ICOConverter is a great website for it.

After you download the icon in .icon extension, just copy it to the pages/static folder of your project and add 1 line to the index.html template:

<link rel="shortcut icon" href="{% static 'favicon.ico' %}"/>

Conclusion

In this story I’ve covered:

how to create a Django project
Django's apps concept
how Django uses urlpatterns to find your view
how to write HTML for your landing page
how to style your landing page so that it looks good
and the cherry on top of the cake — your icon

This is the first part of the series of articles about the MuN. Stay tuned for part 2. You can find the code of this project, as well as my other projects, on my GitHub page. If you liked this article you can follow me and if you didn’t — you can leave an angry comment down below.

What to do when you need a web app quickly

Nikita Kholin — Sun, 04 Nov 2018 16:05:42 +0000

Recently, I needed a piece of software where you can input some numbers, validate and process them, and print some result. Since I needed to do scientific computing with it I decided to use Python with NumPy and SciPy and because I needed validation I decided that HMTL input validations would be the easiest to use.

Then I had a choice from a ton of Python web frameworks and since “microframework” sounded like exactly what I need and I’ve heard the name before I decided to give Flask a try.

Getting started

All you need to get started after you’ve installed Flask is only 1 python file.

from flask import Flask
app = Flask(__name__)

@app.route("/")
def index():
    return "Hello there!"

Start it with a simple command:

FLASK_APP=app.py FLASK_ENV=development flask run

That’s all you need to render a simple string on the “/” page. But who the fuck needs to render a string? Even if all you need is only to render a string it would look awful without some CSS. So we’ll go deeper and get into rendering templates.

Rendering templates

In case you want to render a template you can use a built-in method called, you guessed it, render_template.

from flask import Flask, render_template
app = Flask(__name__)

@app.route("/")
def index():
    return render_template('index.html')

You’ll also need to create index.html page in /templates folder.

<!doctype html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <meta name="viewport" content="width=device-width, initial-scale=1">
  <title>Computer math algorithms</title>
</head>
<body>
  <h1>Computer math algorithms</h1>
  <ul>
    <li><a href="/equations">System of linear equations</a></li>
  </ul>
</body>
</html>

As you can see, I’ve added a link to /equations, but if you click the link now all you’ll see is “404 Not Found” page. To fix this you’ll need to add another route and another template the same way we did before.

@app.route("/equations")
def equations():
    return render_template('equations.html')

The first form

The current project is going to solve a system of linear equations. First of all, we need to know how many equations there are. To get that number we can use a form with one number input.

<form action="/equations">
  <label for="equations_number">Number of equations:</label>
  <input type="number" name="equations_number" required min="1" max="100">
  <br>
  <input type="submit" value="Submit">
</form>

By the way, for some simple CSS I recommend reading a great website. To find CSS used there you can use developers tools.

This form can’t be submitted without number of equations because of the required attribute and on the input tag. Also, the number cannot be less than 1 and more than 100 because of min and max attributes respectively.

The number could be bigger than 100, but the website won’t be usable (it’s not really usable after 10 already). In that case, I would add import from some sort of spreadsheet file.

After entering any valid number, click on “Submit” button and a parameter will be added to the URL /equations?equations_number=2.

Dealing with arguments

To read that parameter on the backend request.args can be used like so:

from flask import request

@app.route("/equations")
def equations():
    equations_number = request.args.get('equations_number', type = int)
    return render_template('equations.html', equations_number=equations_number)

Here we’re passing a name of the needed parameter to request.args.get and a type so that it converts to integer. equations_number argument is then passed to render_template so that it is available in the template.

Is this a system of linear equations?

All we need for the system of equations is a matrix of coefficients A and a column vector b so that the equation would be Ax=b. Example:

x + 2y = 3
4x + 5y = 6

In this case, we would have the following A and b:

A = [[1, 2], [4, 5]]
b = [[3], [6]]

Making a form with a two-dimensional array input

Right now we need a form for A and b.

<form action="/equations">
  {% for i in range(equations_number) %}
    {% for j in range(equations_number) %}
      <label for="a">A[{{i+1}}, {{j+1}}]:</label>
      <input type="number" name="a" class="matrix-number matrix-number--a" required>
    {% endfor %}
    <span class="b">
      <label for="b">b[{{i}}]:</label>
      <input type="number" name="b" class="matrix-number" required>
    </span>
    <br>
  {% endfor %}
  <input type="submit" value="Solve">
</form>

What this code does is it creates a form and equations_number times renders input for each equation. And each equation needs equations_number coefficients (that’s what for loop with j is for) and one number after the equals sign (b).

Another interesting thing to note here is that input with name="a" gets rendered equations_number^2 times and equations_number times for name="b". Flask supports this types of forms and to read data from them request.args.getlist can be used like so:

b = request.args.getlist('b', type=int)

We could do the same for the A array, but we need it to be two-dimensional. This is where NumPy comes in handy with a reshape method. All you need to provide for it is an array and a tuple of the shape you want.

import numpy as np
shape = (len(b), len(b))
A = request.args.getlist(‘a’, type=int) # 1-dimensional
A = np.reshape(A, shape)                # 2-dimensional

Conditional rendering

I would like to hide equations number form after it is submitted. To do that we can use “conditional rendering”. It’s pretty straightforward:

{% if equations_number %}
  <form action="/equations">... form for `A` and `b` ...</form>
{% else %}
  <form action="/equations">... form for `equations_number` ...</form>
{% endif %}

Finally solving the system

Once we have all the coefficients we can solve the system of equations. I could try to bother with how you can invert matrix A and multiply it with a vector b, but I won’t. Today we’ll use NumPy once again. This library has a function for exactly what we want — solving a system of linear equations and it is called numpy.linalg.solve. All it needs is our matrix A and b, which we already have.

x = np.linalg.solve(A, b)

This is where things get a bit tricky. Since we’re using this one action for everything related to solving the system we have add some ifs. Here’s the final code:

equations_number = request.args.get('equations_number', type = int)
b = request.args.getlist('b', type=int)
A = np.array(request.args.getlist('a', type=int)).reshape(len(b), len(b))
x = np.array([])
if len(b) > 0:
    x = np.linalg.solve(A, b)
return render_template('equations.html', equations_number=equations_number, A=A, b=b, x=x)

Let’s check it with some screens we’re using this action for:

Start: equations_number is None, and all other variables are empty arrays
Number of equations is entered: equations_number is present, but all other variables are still empty arrays
A and b are entered: equations_number is None again, b is an array, A is a 2D array and x is an answer for the system of equations

With that we can render our final form:

{% if x.size > 0 %}
  <p>A = {{ A }}</p>
  <p>B = {{ b }}</p>
  <p>x = {{ x }}</p>
{% elif equations_number %}
  <form action="/equations">
    {% for i in range(equations_number) %}
      {% for j in range(equations_number) %}
        <label for="a">A[{{i+1}}, {{j+1}}]:</label>
        <input type="number" name="a" class="matrix-number matrix-number--a" required>
      {% endfor %}
      <span class="b">
        <label for="b">b[{{i}}]:</label>
        <input type="number" name="b" class="matrix-number" required>
      </span>
      <br>
    {% endfor %}
    <input type="submit" value="Solve">
  </form>
{% else %}
  <form action="/equations">
    <label for="equations_number">Number of equations:</label>
    <input type="number" name="equations_number" required min="1" max="100">
    <br>
    <input type="submit" value="Submit">
  </form>
{% endif %}

Areas of improvement (homework?)

Separate views and actions for different screens of equation
A base layout all views
Form accessibility for bigger inputs and prettier output

Conclusion

In this story I’ve covered:

Setting up your Flask application
Rendering templates
Making HTML forms
How to read arguments with Flask (regular, arrays, and even 2-dimensional arrays)
Conditional rendering
How a system of linear equations can be represented and solved with NumPy

You can find the full code of this project, as well as my other projects, on my GitHub page. If you liked this article you can follow me and if you didn’t — you can leave an angry comment down below.

This article was originally posted on my blog.