DEV Community: Gagan

Multi-process vs Multi-threading

Gagan — Thu, 26 Jan 2023 05:43:50 +0000

Multi-process refers to the execution of multiple processes at the same time, where each process has its own memory space and runs independently of the others. This allows for true parallelism and can be useful for tasks that are computationally intensive or need to run concurrently for other reasons. However, communication between processes can be more difficult and resource-intensive.

Multi-threading, on the other hand, refers to the execution of multiple threads within a single process. These threads share the same memory space and can easily communicate with each other. This makes inter-thread communication more efficient and can be useful for tasks that are more IO-bound. However, true parallelism may not be achieved as multiple threads may not be executed simultaneously on different cores.

In terms of design, SQL databases are often used for transactional data where ACID (Atomicity, Consistency, Isolation, Durability) properties are required. SQL databases are well suited for complex queries and reporting. NoSQL databases, on the other hand, are more suited for storing large amounts of unstructured or semi-structured data, scalability and performance under high loads.

In terms of use case, SQL databases are often used in enterprise applications, online transaction processing systems and data warehousing applications. NoSQL databases are often used in real-time big data applications, content management systems, and other web-scale applications.

Multi-process	Multi-thread
Each process has its own memory space and resources	Threads share the same memory space and resources

Inter-process communication can be more complex and slower	Inter-thread communication is faster and simpler

Can take advantage of multiple CPUs or cores	Limited by the number of cores available
Can handle multiple tasks with different priorities more effectively	Prioritization of threads may be more difficult
More memory overhead	Less memory overhead
Better isolation and protection against errors or crashes	Error or crash in one thread can affect other threads

SQL vs NoSQL

Gagan — Thu, 26 Jan 2023 05:36:31 +0000

SQL databases, also known as relational databases, use a structured query language (SQL) for defining and manipulating the data. The data is organized into tables, with each table consisting of rows and columns. SQL databases are best suited for situations where the data has a clear structure and relationships between tables need to be defined. Examples of SQL databases include MySQL, PostgreSQL, and Microsoft SQL Server.

NoSQL databases, on the other hand, do not use SQL and do not have a fixed schema. Instead, they use a variety of data models such as key-value, document, columnar, and graph. NoSQL databases are often used in situations where the data structure is not clear or may change frequently, such as in big data and real-time applications. Examples of NoSQL databases include MongoDB, Cassandra, and Redis.

The main advantage of SQL databases is their ability to handle complex queries and transactions. They also provide a high level of data consistency and are suitable for applications that require strict data integrity. On the other hand, NoSQL databases are more flexible and can handle a larger volume of unstructured data. They also provide better scalability and are suitable for applications that require high performance and availability.

In general, SQL databases are used for transactional systems, while NoSQL databases are used for data warehousing and big data analytics. SQL databases are also used in applications where data consistency is important, while NoSQL databases are used in applications where performance and scalability are more important.

Python Global Interpreter Lock (GIL)

Gagan — Thu, 26 Jan 2023 05:32:44 +0000

The Global Interpreter Lock (GIL) is a mechanism used in the CPython implementation of the Python programming language to ensure that only one thread executes Python bytecode at a time. This lock is necessary because CPython, the reference implementation of Python, is not thread-safe, meaning that multiple threads can potentially interfere with each other and compromise the integrity of an application.

The GIL has both pros and cons. One of the main advantages of the GIL is that it simplifies memory management and object lifetime, making it easier for developers to write correct and stable Python code. Additionally, the GIL can also improve performance for single-threaded programs or programs that spend most of their time performing I/O operations.

On the other hand, the GIL can also have a significant negative impact on the performance of multi-threaded Python programs, particularly those that perform CPU-bound operations. Because only one thread can execute Python bytecode at a time, multiple threads may spend a lot of time waiting for the GIL to be released, leading to decreased performance and increased CPU utilization.

Here is an example of how the GIL can affect the performance of a multi-threaded Python program:

import threading

def count():
    x = 0
    for i in range(1000000):
        x += 1

threads = []
for i in range(10):
    thread = threading.Thread(target=count)
    thread.start()
    threads.append(thread)

for thread in threads:
    thread.join()

print("Done.")

In this example, the count function is called by 10 different threads, each of which increments a shared variable x 1 million times. Due to the GIL, only one thread can execute the function at a time, which may lead to decreased performance and increased CPU utilization.

In conclusion, the Global Interpreter Lock (GIL) is a mechanism used in the CPython implementation of Python to ensure that only one thread executes Python bytecode at a time. While it can simplify memory management and improve performance for single-threaded programs or programs that spend most of their time performing I/O operations, it can also have a significant negative impact on the performance of multi-threaded Python programs, particularly those that perform CPU-bound operations.

Domain Name System

Gagan — Fri, 20 Jan 2023 07:56:04 +0000

DNS stands for Domain Name System. It is a hierarchical and decentralized naming system for computers, services, or any resource connected to the Internet or a private network. It translates human-friendly domain names, such as www.example.com, into IP addresses, such as 192.0.2.1, that are used by computers to identify and locate resources on the Internet.

DNS works by resolving domain names to IP addresses through a series of queries. When a user types a domain name into their web browser, their computer sends a query to a DNS resolver, which is typically provided by their Internet service provider (ISP). The resolver then sends a query to a DNS server, which is responsible for the top-level domain (TLD) of the domain name, such as .com or .org. If the DNS server is not able to resolve the domain name, it sends a query to the next level of DNS servers, until the correct IP address is found.

To reserve a name for your website, you need to register a domain name. This can be done through a domain name registrar, which is an organization that manages the reservation of domain names. When you register a domain name, you are essentially renting the use of that domain name for a specified period of time, usually one year. Once you have registered a domain name, you can then point it to your website's IP address by configuring DNS settings for your domain.

There are also different types of domain names available, such as country code TLDs (ccTLDs) and generic TLDs (gTLDs) .ccTLDs are two-letter top-level domains and are specific to a country or territory. gTLDs are more general and include popular extensions such as .com, .net, and .org.

You can choose a registrar to register your domain name and also can check the availability of the domain. Once you have chosen the registrar and the domain name is available, you can buy it and point it to the IP address of your website and configure it according to your needs.

Bloom filters

Gagan — Fri, 20 Jan 2023 07:52:44 +0000

A Bloom filter is a probabilistic data structure used to test whether an element is a member of a set. The Bloom filter was invented by Burton Howard Bloom in 1970. It is a space-efficient data structure that is used to test whether an element is a member of a set or not. It is used in many applications where it is necessary to quickly determine whether an element is in a set or not, without having to store all of the elements of the set.

A Bloom filter is implemented as a bit array of m bits, all initially set to 0. It also has k hash functions, each of which maps or hashes an element to one of the m bits in the array. To add an element to the set, the element is passed through each of the k hash functions, and the resulting bits in the array are set to 1. To check if an element is in the set, the element is passed through each of the k hash functions, and if any of the resulting bits in the array are 0, the element is definitely not in the set. If all of the resulting bits are 1, the element may be in the set, but it could also be a false positive due to the probabilistic nature of the Bloom filter.

Bloom filters are used in many applications, such as in spell-checking, network routers, and databases, where it is necessary to quickly determine whether an element is in a set without having to store all of the elements in the set. They are also used in bioinformatics to quickly identify whether a DNA sequence has been previously observed.

Here's an example of a simple implementation of a Bloom filter in Python:

import mmh3

class BloomFilter:
    def __init__(self, size, hash_count):
        self.size = size
        self.hash_count = hash_count
        self.bit_array = [0] * size

    def add(self, item):
        for seed in range(self.hash_count):
            result = mmh3.hash(item, seed) % self.size
            self.bit_array[result] = 1

    def check(self, item):
        for seed in range(self.hash_count):
            result = mmh3.hash(item, seed) % self.size
            if self.bit_array[result] == 0:
                return "Nope"
        return "Maybe"

bf = BloomFilter(32, 5)
bf.add("hello")
bf.add("world")
print(bf.check("hello")) # prints "Maybe"
print(bf.check("hi")) # prints "Nope"

In this example, the BloomFilter class has a constructor that takes in two arguments: the size of the bit array and the number of hash functions to be used. The add method takes an item and adds it to the filter by passing it through each of the hash functions and setting the bits.

Python Memory leaks

Gagan — Fri, 20 Jan 2023 07:48:35 +0000

A memory leak in Python occurs when the program dynamically allocates memory for an object and then fails to deallocate it, even though it is no longer needed. This can cause the program to consume an increasing amount of memory over time, eventually causing the program to slow down or crash.

There are several ways that memory leaks can occur in Python. One common cause is when a program creates a reference cycle. A reference cycle occurs when two or more objects reference each other, causing them to be kept in memory even though they are no longer needed.

Here's an example of a simple Python program that demonstrates a memory leak caused by a reference cycle:

import gc

class Foo:
    def __init__(self):
        self.bar = None

foo1 = Foo()
foo2 = Foo()
foo1.bar = foo2
foo2.bar = foo1

gc.collect()
print(gc.garbage)  # will print [<__main__.Foo object at 0x...>]

In abve example, the two Foo objects, foo1 and foo2, reference each other, creating a reference cycle that the garbage collector is unable to break. This means that the objects will not be deallocated, and the program will continue to consume more and more memory.

Another way of leaking memory in python is by using global variables and not cleaning them properly or not re-initializing them. Also in some cases, using too many list comprehensions, or using recursion instead of iteration can lead to memory leaks.

To avoid memory leaks, it's important to keep track of the references to objects and make sure that they are properly deallocated when they are no longer needed. This can be achieved by using the del keyword, or by using the gc.collect() function to manually invoke the garbage collector. Also, it's important to use Python's garbage collector, which automatically frees memory that is no longer in use.

GitHub Password Authentication Deprecation

Gagan — Sun, 16 Jan 2022 14:00:01 +0000

As per GitHub' announcement, December 2020, account passwords will no longer be supported to authenticate Git operations beginning on August 13, 2021. In other words, password authentication has been deprecated.

If you have a tool or are heavily dependent on git cli for everyday workflows, you will be impacted by this. Please do wait until the last minute to update your GitHub auth method; do it today and get acquainted with the new workflow.

The two Github authentication methods supported will be:

Please read more on the above to set it up.