DEV Community: Manav

Context Managers in Python.

Manav — Fri, 29 Jan 2021 12:25:21 +0000

This post is inspired from a talk at python pune january meetup by Pradhavan.

What are Context Managers?

Here's what Python's official documentation says:

A context manager is an object that defines the runtime context to be established when executing a with statement. The context manager handles the entry into, and the exit from, the desired runtime context for the execution of the block of code. Context managers are normally invoked using the with statement, but can also be used by directly invoking their methods. – Python Docs

But that's just covering all the bases. Let's understand a simplified version.

Programmers work with external resources from time to time, like files, database connections, locks etc. Context managers allow us to manage those resources by specifying:

What to do when we acquire the resource, and
What to do when the resource gets released

Why do we need Context Managers?

Consider the following example:

for _ in range(100000):
    file = open("foo.txt", "w")
    files.append(f)
    file.close()

Notice that we're calling the close() method to ensure that the file descriptor is released every time. If we didn't do that, our OS would run out of its allowed limit to open file descriptors eventually.

However, we write a more pythonic version of the above code using context manager:

for _ in range(100000):
    with open("foo.txt", "r") as f:
        files.append(f)

Here open("foo.txt", "r") is the context manager that gets activated using the with statement. Notice that we didn't need to explicitly close the file, the context manager took care of it for us. Similarly there are other predefined context managers in Python that makes our work easier.

Can we define our own Context Manager?

Yes. There are two ways to define a custom context manager:

Class based definition.
Function based definition.

Class Based Context Managers

Let's continue with our file example and try to define our own context manager which will emulate open().

files = []

class Open():
    def __init__(self, filename, mode):
        self.filename = filename
        self.mode = mode

    def __enter__(self):
        self.open_file = open(self.filename, self.mode)
        print("Enter called")
        return self.open_file

    def __exit__(self, *args):
        print("Exit Called")
        self.open_file.close()

for _ in range(100000):
    with Open('foo.txt', 'w') as f:
        files.append(f)

The __enter__ method tells us what to do when we acquire the resource, i.e., giving us a file object.
The __exit__ method defines what to do when we're exiting the context manager, i.e., closing the file.
You can see how both __enter__ and __exit__ are called with every loop.

Handling Errors

How do we handle FileNotFoundError with python's open()

try:
    with open("foo.txt", "r") as f:
        content = f.readlines()
except FileNotFoundError as e:
    print("Hey, file isn't there. Let's log it.")

Such a basic error handling code that needs to be every time you open a file. Let's try to DRY it with our custom context manager.

class Open():
    def __init__(self, filename, mode):
        self.filename = filename
        self.mode = mode

    def __enter__(self):
        print("Enter called")

        try:
            self.open_file = open(self.filename, self.mode)
            return self.open_file
        except FileNotFoundError as e:
            print("Hey, file isn't there. Let's log it.")

    def __exit__(self, exc_type, exc_value, exc_traceback): #notice the parameters
        print("Exit Called")
        if(exc_type is None):
            self.open_file.close()
            return True
        else:
            return True

with Open("foo.txt", "r") as f:
    content = f.readlines()

Changes in __exit__

exc_type is type of error Class which you'll get while handling errors in __enter__ (AttributeError in this case).
exc_value is the value of the error which you'll get while handling errors in __enter__.
exc_traceback is the traceback of the error which you'll get while handling errors in __enter__.
We're returning True to suppress the error traceback (not to be confused with exc_traceback parameter).

Another Real World Example

class DatabaseHandler():
    def __init__(self):
        self.host = '127.0.0.1'
        self.user = 'dev'
        self.password = 'dev@123'
        self.db = 'foobar'
        self.port = '5432'
        self.connection = None
        self.cursor = None

    def __enter__(self):
        self.connection = psycopg2.connect(
            user=self.user,
            password=self.password,
            host=self.host,
            port=self.port,
            database=self.db
        )
        self.cursor = self.connection.cursor()
        return self.cursor

    def __exit__(self, *args):
        self.cursor.close()
        self.connection.close()

Function Based Context Managers

Function based context management is done by using a lib called contextlib, through which we can change a simple generator function into a context manager. Here's what a typical blueprint looks like:

from contextlib import contextmanager

@contextmanager
def foobar():
    print("What you would typically put in __enter__")
    yield {}
    print("What you would typically put in __exit__")

with foobar() as f:
    print(f)

contextmanager decorator is used to turn any generator function into a context manager.
yield work as a separater between __enter__ and __exit__ parts of the context manager.

Handling files

from contextlib import contextmanager

@contextmanager
def open_(filename, mode):

    print("SETUP")
    open_file = open(filename, mode)

    try:
        print("EXECUTION")
        yield open_file

    except:
        print("Hey, file isn't there. Let's log it.")

    finally:
        print("CLEAN-UP")
        open_file.close()


with open_("somethign.txt", "w") as f: #notice the mode
    content = f.readlines() #you cannot read on write mode

We wrap yield in a try block because we don't know what the user is going to do with the file object. They might try to use it in a way that it's not intended to (as shown above).

Database Connections

from contextlib import contextmanager

@contextmanager
def database_handler():
    try:
        host = '127.0.0.1'
        user = 'dev'
        password = 'dev@123'
        db = 'foobar'
        port = '5432'
        connection = psycopg2.connect(
            user=user,
            password=password,
            host=host,
            port=port,
            database=db
        )
        cursor = connection.cursor()
        yield cursor

    except:
        print("Hey, file isn't there. Let's log it.")

    finally:
        cursor.close()
        connection.close()

Resources

We have just covered just an introduction to context managers, but I feel that it's just the tip of the iceberg and there are many interesting use cases for it. Here are some interesting links that I found:

@property in Python.

Manav — Thu, 14 Jan 2021 16:27:53 +0000

@property is a popular keyword in python. In this post you'll learn how can we use it to write more pythonic code and refactor any non pythonic code we had.

Let's consider an hypothetical number class which only allows numbers that are greater than 23 and less than 5. A typical OOPs program for it would look like this:

#python3.8
class HypotheticalNumber():

    def __init__(self, num=0):
        self.set_num(num) 

    def get_num(self):
        return self.num # notice public scope

    def set_num(self, num):
        if(num > 23 or num < 5):
            self.num = num
        else:
            self.num = 0

some_number = HypotheticalNumber(4)
print(some_number.get_num()) #4
print(some_number.num) #4
some_number.set_num(5) # sets to 0
some_number.num = 5 # sets to 5

As we can see, the above implementation has some major holes in it. Not only we're able to access our hypothetical number by simply referring to some_number.num directly, we can also change it by direct access and break the core rule that our class is based upon, i.e. assigning a value equal to 5.

There's a pythonic way to do it.

@property is a decorator that handles getting, setting and deleting of class variables, the way it was meant to be in Python. A pythonic program for the above scenario would look like:

class HypotheticalNumber():

    def __init__(self, num=0):
        self.num = num

    @property
    def num(self):
        return self.__num

    @num.setter
    def num(self, num):
        if(num > 23 or num < 5):
            self.__num = num
        else:
            self.__num = 0

    @num.deleter
    def num(self):
        del self.__num

some_number = HypotheticalNumber(4)
print(some_number.num) #4
some_number.num = 5 # sets to 0

Let's take it step by step.

@property is used to create a getter like function which has the same name as the class variable itself.
@num.setter is used to create a setter like function which defines what to do while setting the variable.
@num.deleter is used to create a deleter like function which defines what to do while deleting the varible.
Notice that all the functions have same name. It is necessary for the program to work.

What if my project contains code with getter and setter methods instead of property?

Well, you can always rewrite the code, or, there's another way to refactor it. It might save some time for you. Instead of adding @property as a decorator, we can just add the getter and setter methods to the property function.

#python3.8
class HypotheticalNumber():

    def __init__(self, num=0):
        self.set_num(num) 

    def get_num(self):
        return self.__num # notice dunder

    def set_num(self, num):
        if(num > 23 or num < 5):
            self.__num = num
        else:
            self.__num = 0

    num = property(get_num, set_num)

some_number = HypotheticalNumber(4)
print(some_number.get_num()) #4
print(some_number.num) #4
some_number.set_num(5) # sets to 0
some_number.num = 5 # sets to 0

As you can see, we just needed to change the variables and add a special line
num = property(get_num, set_num)
that secured the holes we had in our initial code and saved us a lot of time!

3 Basic JavaScript Concepts That Are Important.

Manav — Thu, 07 Jan 2021 16:32:07 +0000

Learning javascript is weird. I want to share 3 things that I learnt today in javascript.

Hoisting

Hoisting is a mechanism where variable or function declarations are moved to the top of their scope before they're executed. That means, if we do this:

console.log(greeter);
var greeter = "say Hello!";

It'll be interpreted as this:

var greeter;
console.log(greeter); //undefined
greeter = "say Hello!";

null vs undefined

null is a value, just like 2, 3.5 or true. Setting a variable to null means that you have declared a variable and decided to give the value - nothing, to it.
undefined on the other hand, means you have declared a variable but not given it any value yet.

Interestingly, null loosely equalizes to undefined but there are a few gotchas!

console.log(null == undefined); //true
console.log(null === undefined); //false
console.log(null >= 0); //true
console.log(null <= 0); //true
console.log(undefined <= 0); //false
console.log(undefined >= 0); //false

`==` vs `===`

This is a simple one. == means loose equality. It's used when you want to know if two values are equal if they'd have same type. For example:

console.log(0 == false); //true
console.log(1 == 1.0); //true
console.log(1 == '1'); //true

=== however, checks for strict equality. It means, it'll only return true IF the two values are absolutely same. For example:

console.log(0 === false); //false
console.log(1 === 1.0); //true, it's a bad idea to compare floats anyway.
console.log(1 === '1'); //false

That's all folks!

How I Used Airtable To Create A Book Lending Service For Free

Manav — Wed, 16 Sep 2020 21:14:40 +0000

I moved to my hometown due to covid-19. It's a small town in the foothills of Himalayas, called 'Kotdwara'. The view in the article cover is the one I grew up with.

I am not a voracious reader, but I try to read a book a month, which, over the years, have accumulated to ~100 books. But now that I've read them (and re-read some), what do I do with them?

I always wanted to setup a local library in Kotdwara. Growing up, there was no Amazon. There wasn't even a decent book shop where you could get some novel (isn't even one now). But, I don't want to be a book shop owner. Nor do I have a space big enough where people can come and read. But I do, however, have books.

What if I created an online book lending service?

The idea was exciting, nobody was doing it in Kotdwara. May be there's no market for it, but I won't know for fact if I don't try.

So I decided to try it. My sister and I came up with the pricing, service flow and a poster that can summarize what we're trying to do.

Now all I needed to figure out was how to build that catalogue? I needed an interface that:

didn't take much time to setup & maintain
was web based (platform independent), and
Free

... I knew Airtable from a recent freelance gig and I knew there must be something that could help me.

I logged into my Airtable account, and checked if they have something that could help me. To my surprise, there was a template specially for this type of situation.

I quickly edited the template to my requirements and started filling the data for the books. I was afraid about my storage limits and affordability if I wanted to expand. Turns out, I could store upto 1200 records and 2GB worth of attachments in the free plan.

Next, I wanted a way to somehow show people if a particular book is available for borrowing or somebody has already borrowed it. Technically, I required a trigger mechanism. This feature has been added in the form of Automations in Airtable platform which makes it really easy to create a trigger and do 'action(s)' in response to the triggers. Here's a screenshot of one of the automations.

Although there's a limit of 100 runs for these actions per month, it'd mean that I could only use it for ~50 book lendings/returns but that was enough for a start.

Now the only thing left was to create a view that our borrowers would see. Airtable has gallery view feature where we can share whole, or part of our sheet, as a gallery. I used it to finally create our catalogue, have a look.

*Thanks for reading, please follow me for more little stories like this one. *

How To Create A CSS Heart

Manav — Sun, 13 Sep 2020 20:48:03 +0000

Share your Google Season of Docs Experience

Manav — Fri, 15 May 2020 20:05:03 +0000

We've just hit the 'Technical writer exploration' milestone in the annual Google's Season of Docs. I am interested in participating but before that I want to know what I am getting myself into.

Anyone who was previously selected for GSOD, can you share your experience? Here are some focus points:

How much time did you end up spending in a week?
Did it open further networking opportunities for you?
How long did the project last for you?
What were the low points during your gsod journey?

How to Control Tor Exit Nodes with Python

Manav — Fri, 08 May 2020 06:39:23 +0000

As you know, Tor routes requests from atleast 3 nodes(by default) before querying the destination server. In this tutorial we’re going to learn how to control exit location (last node) in Tor.

Please visit the following url for the tutorial:

https://sigmapie8.github.io/learning-stem/controlling_exit.html

Liked the tutorial? Tell me in comments!

How To Use Requests With Tor

Manav — Wed, 06 May 2020 15:43:40 +0000

What's Tor? Software that helps you browse the internet anonymously (almost).

What's stem? Python library to interact with Tor.

In this tutorial we’re going to learn how to use the famous requests library to get resources from internet via Tor network. That means, we’ll be using tor as a proxy between us and the internet.

https://sigmapie8.github.io/learning-stem/requests_with_tor.html

Tell me if you liked it :)

Would You Decline A Project Due To Ethics?

Manav — Mon, 30 Mar 2020 11:22:25 +0000

I used to work in a company that believed in intensive tracking. Our computers had productivity trackers which used to log our every activity and of the 8 hours(out of 9) that we were billed, we had to show 8 hours of productivity.
Life was hell. We ended up sitting, moving mouses and pushing keys. We used to stay in office for 10-11 hours so that we could complete our fill of productivity, even when there wasn't any work to do. I used to curse the companies who made these kind of tracking software.

My question to you is, would you build such kind of software for money if you knew how is it going to be used? What're your thoughts about it.

Compiled VS Interpreted: A Code Example

Manav — Mon, 16 Mar 2020 04:44:59 +0000

I've been writing code in python (interpreted language) for some time and as expected I, unconsciously, got habitual of writing interpreted code and python's lazy evaluation. For example, the following code works in python:

if(1 == 2):
    print(a)

It doesn't matter if the variable a has been declared or not, the code works because python never executes the print statement until the if condition is true.

On the other hand, when you try to do it in a compiled language like nim, it fails, because as expected of a compiler, it evaluates everything it can during compilation. So I get the error:

if(1 == 2):
    echo a #ERROR: undeclared identifier: 'a'

The above example seems extremely simple, but you can end up with this kind of gotcha anywhere, like I did when writing the following code in nim:

type 
    htag* = object
        text: string
        level: int
        name: string

    atag* = object
        text: string
        link: string
        name: string



proc renderTag(tag: object): string =
    echo tag
    if(tag.name == "h"):
        return "<h" & $tag.level & ">" & tag.text & "</h" & $tag.level & ">"

    elif(tag.name == "a"):
        return "<a href='" & tag.link & "' target='_blank'>"

    else:
        return "nothing"

Here, if I declare one kind of object, I'll get an undeclared field error with the other.

Learning Nim: Adding A Package to Nim Ecosystem

Manav — Fri, 28 Feb 2020 17:59:54 +0000

I recently published my first package in the Nim ecosystem. In this post I am going to show how I did it.

The package I published is called luhny it is the Luhn's Algorithm's implementation in Nim. Using this, you can validate any credit card number as Luhn's Algorithm is the standard for credit card validation.

But why would I make a package out of it?

I created it for two reasons:

I wanted to know how to publish a package
I want to add a generate valid numbers functionality that people may be able to use in the future.

Step 1: Make a Repository

No matter how small or how easy of a thing you're building. Good habits help you go a long way.

git init <name of your package>

Step 2: Repository Structure

You're free to choose whatever structure you want. I saw a couple of packages on github and decided to go with the following structure:

luhny
|_____________ .git
|_____________ .gitignore
|_____________ luhny
               |________ luhny.nim
|_____________ tests
               |________ test.nim
               |________ config.nims #notice the extra 's'
|_____________ readme.md
|_____________ luhny.nimble

You can know about the updated package structure here

Step 4: Tests

Again, habits matter. No matter how small or easy your code is, you got to test it. I tried to use nim's unittest module.

#test.nim
# Blueprint for a basic test suite

import unittest
import luhny # code that I want to test

#A test may or may not contain more than one test
suite "<name of your test suite>":
    echo "Initializing"

    # use it if you want to run any code before every test
    setup:
        echo "setup code executed"

    # use it if you want to run any code after every test
    teardown:
        echo "teardown code executed"

    #test1
    test "<name of your test that your want to display>":
        check(<function you want to test>() == <its expected outcome>)

    #for example
    test "Wrong input test":
        check(check_number("12345") == false)

    test "Correct input test":
        check(check_number("4532196437678779") == true)

    test "Single 0":
        check(check_number("0") == true)

    test "Empty String":
        check(check_number("") == true)

    # Message indicating end of tests
    echo "Tests Completed"

But how does test.nim find luhny.nim? They are in different folders.

Yes, for that we use nimscript. Remember the config.nims file? We'll use that to switch the path.

#config.nims
switch("path", "$projectDir/../luhny") 
# changed the path of the current directory from tests to luhny.

... and that's how tests found luhny. Although, there's another (official) way to do it.

After running your tests and getting a satisfactory output, we can work towards making it an actual package by adding a .nimble file.

Step 5: Add a `.nimble` File

Go to your project directory, that is, the top directory that has readme file. There you execute the following command:

nimble init

It will be followed by a series of questions like

What's the package version
what's the author name
what's license and dependencies etc

After filling the details, you can even try to install the package by using nimble install inside the project directory.

Step 6: Uploading to github

Remember that till now, everything is going on in your local repository. Now it's time to push the changes to github so that we can share our package with the world.

git remote add origin git@github.com:<your-user-name>/<you package name>.git
git push -u origin master

Step 7: Telling Nimble About Your Package

There's a separate package repository of Nim packages hosted on github that covers this step and is fairly comprehensive about it.

Create a PR and your package would be added to the Nim's Ecosystem when the PR is accepted.

If you learnt anything from this post, please hit ❤️ and share it with other people!

Self Learning via Online Resources VS Getting a Degree

Manav — Sun, 23 Feb 2020 17:13:33 +0000

I want to know the thoughts of DEV community about how likely is anyone to succeed with self learning vs paying for a college degree.