DEV Community: Mohd. Shafikur Rahman

Understanding Overcommit Memory in Linux Kernel

Mohd. Shafikur Rahman — Sat, 30 Mar 2024 10:06:09 +0000

Overcommit memory, a crucial feature in the Linux kernel, enables processes to allocate memory beyond the system's physical limits. This mechanism relies on the premise that most applications still need to utilize the memory they request fully. Let’s investigate how overcommit memory works and its system performance and stability implications.

When a process requests memory, the Linux kernel reserves address space without immediately allocating physical memory. Instead, it defers physical memory allocation until the memory is accessed. This approach allows the system to appear to have more available memory than it physically possesses.

The vm.overcommit_memory setting in the Linux kernel governs the behavior of memory overcommitment. It offers three distinct values:

0 (default): This setting implements heuristic memory overcommitment. It permits memory allocation to surpass the sum of available physical memory and swap space but within a configurable limit defined by vm.overcommit_ratio.
1: Enabling full overcommitment, this setting allows memory allocation to exceed the total amount of physical RAM and swap space. While advantageous for specific workloads, it heightens the risk of encountering out-of-memory (OOM) errors when the system exhausts physical and swap space.
2: Disabling memory overcommitment: This setting mandates that the kernel allocate memory only if sufficient physical and swap space is available to fulfill the request.

Setting vm.overcommit_memory to 1 facilitates full overcommitment, offering flexibility for memory-intensive applications. However, it necessitates careful monitoring of system resources to prevent OOM errors under heavy loads.

Overcommit memory in the Linux kernel is vital in optimizing memory utilization by allowing processes to allocate memory beyond physical constraints. By understanding the implications of vm.overcommit_memory, system administrators can tailor memory management strategies to suit specific workloads, effectively balancing performance and stability.

Deep Copy and Shallow Copy in Python

Mohd. Shafikur Rahman — Fri, 09 Dec 2022 11:08:36 +0000

Shallow Copy and Deep Copy

In this article, we will learn about shallow copy and deep copy in Python.

In Python, we use the assignment (=) operator to create a copy of an object. However, it doesn't make a new object. Instead, it just shares the reference of the original object to a new variable.
In the following example, the old_list hold value [1, 2, 3] and assign to new variable named new_list by using assignment(=) operator. If we modify the new_list, the old_list will also be affected. You can also see that the old_list and new_list id are the same, i.e., 4350750272.

>>> old_list = [1, 2, 3]
>>> new_list = old_list
>>> new_list[2] = 0
>>> print(old_list)
[1, 2, 0]
>>> print(new_list)
[1, 2, 0]
>>> print(f"id of new_list {id(new_list)}, id of old_list {id(old_list)}")
id of new_list 4350750272, id of old_list 4350750272

Why do we need a deep copy or a shallow copy?

Sometimes we need to keep original values unchanged and only modified in the new ones or vice versa. In Python, we have two ways to do this.

Shallow copy
Deep copy

Python copy Module

Python has a copy module for the deep and shallow copy.

copy.copy(x)
Return a shallow copy of x.

copy.deepcopy(x)
Return a deep copy of x.

exception copy.Error
Raised for module-specific errors.

Shallow Copy

A shallow copy creates new objects that store the original objects' reference.
A shallow copy doesn't create a copy of nested objects where, as it only copies the reference of the nested objects. [1]

Create a copy using shallow copy

>>> import copy
>>> old_list = [[1, 2], [3, 4]]
>>> new_list = copy.copy(old_list)
>>> print(old_list)
[[1, 2], [3, 4]]
>>> print(new_list)
[[1, 2], [3, 4]]

In the above example, we create a nested list and shallow copy the value to another variable named new_list using the copy method, it will create independent objects with the same content.
Now we add some data in old_list.

>>> old_list.append([5,6])
>>> print(old_list)
[[1, 2], [3, 4], [5, 6]]
>>> print(new_list)
[[1, 2], [3, 4]]

Here, we append new list [5, 6] to old_list. The new_list contains references of original nested objects stored in old_list. The new sublist [5, 6] was not copied in new_list. But, if you change any nested objects in the old_list, the changes also reflect in the new_list because both lists share the reference of the same nested objects.

>>> old_list[1][1] = 'X'
>>> print(old_list)
[[1, 2], [3, 'X'], [5, 6]]
>>> print(new_list)
[[1, 2], [3, 'X']]

Deep Copy

A deep copy creates a new object and recursively adds the copies of nested objects present in the original elements. [1]

Create a copy using deep copy

>>> import copy  
>>> old_list = [[1, 2], [3, 4]]  
>>> new_list = copy.deepcopy(old_list)  
>>> print(old_list)  
[[1, 2], [3, 4]]  
>>> print(new_list)  
[[1, 2], [3, 4]]

If any changes to nested objects of the old_list, the changes are not reflected in new_list.

>>> import copy
>>> old_list = [[1, 2], [3, 4]]
>>> new_list = copy.deepcopy(old_list)  
>>> old_list[1][1] = 'X'
>>> print(old_list)
[[1, 2], [3, 'X']]
>>> print(new_list)
[[1, 2], [3, 4]]

What is the Difference Between Deep and Shallow Copies? [2]

The difference between shallow and deep copying is only relevant for compound objects (objects that contain other objects, like lists or class instances):

A shallow copy constructs a new compound object and then (to the extent possible) inserts references into it to the objects found in the original.
A deep copy constructs a new compound object and then, recursively, inserts copies into it of the objects found in the original.

References:

[1] https://www.programiz.com/python-programming/shallow-deep-copy

[2] https://docs.python.org/3/library/copy.html

Originally published in https://shafik.io/2022/12/deep-copy-and-shallow-copy-in-python

Stack in Python

Mohd. Shafikur Rahman — Tue, 13 Jul 2021 17:21:47 +0000

Stack is a linear data structure that stores data in LIFO (Last In First Out) manners.
Web browser's back button is a stack application.

We perform two major operations in stack that are PUSH and POP.
The PUSH operation adds an element/ value in the stack, and the POP operation removes or deletes an element / value from the stack.

Some basic operations in stack

PUSH - Add an element or value in the stack. The time complexity is O(1)
POP - Remove topmost element or value from the stack. The time complexity is O(1)
TOP - Get the top element or value of stack without removing it. It is also known as PEEK. The time complexity is O(1)
EMPTY - Check if the stack is empty or not. The time complexity is O(1)
FULL - Check if the stack is full. The time complexity is O(1)
SIZE - Check size of the stack. The time complexity is O(1)

Implementation of stack

There are various options to implement stack in Python.
Here, we discuss stack implementation using Python, and it’s modules.
Stack in Python can be implemented by following ways:

list
collections.deque
queue.LifoQueue

Implementation using List

Python list can be used as the stack.
List append() method used for adding elements in stack, which is similar to stack push operation.
List pop() method used for removing elements from stack, which is similar to stack pop() operation.
Python lists have performance issues.
The list push(), pop() operation becomes slow as it grows.
If the list grows and out of a block of memory then python allocates some memory.
This is the reason why Python lists become slow gradually.
The time complexity is O(n).

Let's understand the following example

# Define empty stack
stack = []

# Add element to stack
stack.append(1)
stack.append(2)
stack.append(3)
stack.append(4)
stack.append(5)

print("Display stack after PUSH operation")
print(stack)

print("Remove element from stack")
print(stack.pop())
print(stack.pop())
print(stack.pop())
print(stack.pop())
print(stack.pop())

print("Display stack after POP operation")
print(stack)

Output:

Display stack after PUSH operation
[1, 2, 3, 4, 5]
Remove element from stack
5
4
3
2
1
Display stack after POP operation
[]

In above code, we defined empty list.
Then append some elements in stack, which is similar to stack PUSH() operation.
We also removed the elements using pop() method.
The pop() method returns the last element from the list.

Implementation using collection.deque

The Python collection module provides the deque class.
By using deque, we can create a stack.
The deque is pronounced as the deck which means double-ended queue.
The deque is better than Python list, because it's append and pop operation faster than the Python list.
The time complexity is O(1).

Let's understand the following example

# Define empty stack
from collections import deque

stack = deque()

# Add element to stack
stack.append(1)
stack.append(2)
stack.append(3)
stack.append(4)
stack.append(5)

print("Display stack after PUSH operation")
print(stack)

print("Remove element from stack")
print(stack.pop())
print(stack.pop())
print(stack.pop())
print(stack.pop())
print(stack.pop())

print("Display stack after POP operation")
print(stack)

Output:

Display stack after PUSH operation
[1, 2, 3, 4, 5]
Remove element from stack
5
4
3
2
1
Display stack after POP operation
[]

Implementation using LifoQueue

The Python queue module hase LifoQue, which is the same as stack.
It's have put() method to add element in stack and put() method to remove from stack.

Some basic operations in LifoQueue

put() - Add element to stack. If queue is full, it's wait until space is available.
put_nowait() - Add element to stack. Only enqueue the item if a free slot is immediately available. Otherwise raise the Full exception.
qsize() - Return the approximate size of the queue (not reliable!)
empty() - Return True if the queue is empty, False otherwise (not reliable!)
maxsize - The used to set maximum size of queue. If maxsize is <= 0, the queue size is infinite.
full() - Return True if the queue is full, False otherwise (not reliable!)
get() - Remove and return an item from the queue.
get_nowait() - Remove and return an item from the queue without blocking. Only get an item if one is immediately available. Otherwise raise the Empty exception.

Let's understand the following example

# Define empty stack
from queue import LifoQueue

stack = LifoQueue(maxsize=5)

print(f"Size of stack: {stack.qsize()}")
# Add element to stack
stack.put(1)
stack.put(2)
stack.put(3)
stack.put(4)
stack.put(5)

print(f"Stack is Full: {stack.full()}")
print(f"Size of Stack: {stack.qsize()}")

print("Remove element from stack")
print(stack.get())
print(stack.get())
print(stack.get())
print(stack.get())
print(stack.get())

print(f"Stack is Empty: {stack.empty()}")

Output:

Size of stack: 0
Stack is Full: True
Size of Stack: 5
Remove element from stack
5
4
3
2
1
Stack is Empty: True

Initially stack size is 0. Then we push 5 elements in stack using put() method.
Then, we check stack is full or not. Then, we check stack size.
After that, we remove element from stack using get() method.
Finally, we check does stack is empty or not.

Python stack and threading

We can use Python stack in multi-threaded program. Using list in multi-threading programming can dangerous because it's not thread safe.

The deque is a little complex because its append() and pop() method are atomic, which emans they will not interrupt by the other thread.

So, for build program of Python stack with treading, LifoQueue is better. It uses put() and get() to add and remove the stack element.

Applications of Stack

Although stack is a simple data structure to implement, it is very powerful.
The most common uses of a stack are:

To reverse a word - Put all the letters in a stack and pop them out. Because of the LIFO order of stack, you will get the letters in reverse order.
In compilers - Compilers use the stack to calculate the value of expressions like 2 + 4 / 5 * (7 - 9) by converting the expression to prefix or postfix form.
In browsers - The back button in a browser saves all the URLs you have visited previously in a stack. Each time you visit a new page, it is added on top of the stack. When you press the back button, the current URL is removed from the stack, and the previous URL is accessed.
Function call and return process - When we call a function from one other function, that function call statement may not be the first statement. After calling the function, we also have to come back from the function area to the place, where we have left our control. So we want to resume our task, not restart. For that reason, we store the address of the program counter into the stack, then go to the function body to execute it. After completion of the execution, it pops out the address from stack and assign it into the program counter to resume the task again.
Backtracking Procedure - Backtracking is one of the algorithm designing technique. For that purpose, we dive into some way, if that way is not efficient, we come back to the previous state and go into some other paths. To get back from current state, we need to store the previous state. For that purpose, we need stack. Some examples of backtracking is finding the solution for Knight Tour problem or N-Queen Problem etc.
Memory management - The assignment of memory takes place in contiguous memory blocks. We call this stack memory allocation because the assignment takes place in the function call stack. The size of the memory to be allocated is known to the compiler. When a function is called, its variables get memory allocated on the stack. When the function call is completed, the memory for the variables is released. All this happens with the help of some predefined routines in the compiler. The user does not have to worry about memory allocation and release of stack variables.
Expression Conversion - Converting one form of expressions to another is one of the important applications of stacks.
1. Infix to prefix
2. Infix to postfix
3. Prefix to Infix
4. Prefix to Postfix
5. Postfix to Infix
6. Postfix to Infix
Parenthesis Matching - Given an expression, you have to find if the parenthesis is either correctly matched or not. For example, consider the expression ( a + b ) * ( c + d).

In the above expression, the opening and closing of the parenthesis are given properly and hence it is said to be a correctly matched parenthesis expression. Whereas, the expression, (a + b * [c + d ) is not a valid expression as the parenthesis are incorrectly given.
Used in many algorithms like Tower of Hanoi, tree traversals, stock span problem, histogram problem.
In Graph Algorithms like Topological Sorting and Strongly Connected Components

Conclusion

We have mentioned three methods of implement the stack in Python. The above methods have their own advantages or disadvantages.

We are using stack with the threading, you should use the LifoQueue but make sure about its performance for popping and pushing elements.
But if you are not using threading, use a deque.

We can also use the list to implement the stack but the list can have potential memory reallocation issues. The list and deque are same in the interface, but the deque doesn't memory allocation issues.

Project specific git configuration

Mohd. Shafikur Rahman — Sat, 08 Feb 2020 18:17:58 +0000

Sometimes we need to project specific git user configuration.

How we achieve this?
Just run following command in project directory.

git config --local user.name "AuthorName" 
git config --local user.email "AuthorEmail@mail.com"

Now all commit author will be new name and email for configured project.

That's all!

Intro to CSS Flexbox layouts

Mohd. Shafikur Rahman — Tue, 21 Jan 2020 18:30:15 +0000

The Flex Box Module usually referred to as flexbox, was designed as an one-dimensional layout model.
It make easier to design flexible responsive layout structure without using float or positioning.

Here is a demo to play around about CSS Flex Box Module: https://shafikshaon.github.io/css-flexbox/

The flex container properties are:

flex-direction
flex-wrap
flex-flow
justify-content
align-items
align-content

flex-direction

The main axis defined by the flex-direction property. It defines in which direction the container want to stack the flex items.

The flex-direction has four possible values:

row
row-reverse
column
column-reverse

row: The row value stacks the flex item horizontally but from left to right.

row-reverse: The row value stacks the flex item horizontally but from right to left.

column: The column value stacks the flex item vertically but from top to bottom.

column-reverse: The column value stacks the flex item vertically but from bottom to top.

flex-wrap

Flex box is one dimensional model. So, it is possible to cause our flex items wrap into multiple lines. We should consider each line as new flex container.

flex-wrap property specifies whether items should wrap or not.
Wrap means cover or enclose (someone or something) in paper or soft material.

wrap: The wrap value specifies that the flex items will wrap if necessary.

nowrap: The wrap value specifies that the flex items will not wrap.

wrap-reverse: The wrap value specifies that the flex items will wrap if necessary but in reverse order.

flex-flow

flex-flow is shortland for both the flex-direction and flex-wrap properties

for example

.class {
    flex-flow: row wrap;
}

justify-content

The justify-content property is used to align the items on the main axis where direction determine by flex-direction.

flex-start: flex-start value align flex items at the start of the container.

flex-end: flex-end value align flex items at the end of the container.

center: center value align flex items at the center of the container.

space-around: space-around value align flex items with space before, between and after the lines. It causes equal amount of space on the right and left of each item.

space-between: space-between value align flex items with space between the lines.

space-evenly: space-evenly value align flex items with full-size space on either end.

align-items

The align-items property used for align flex items vertically.

stretch: stretch value stretch the flex items to fill the container. This is why flex items stretch to the height of the tallest one by default.

baseline: baseline value align the flex items by their flex container baselines.

center: center value align the flex items to center of the container.

flex-start: flex-start value align the flex items to top of the container.

flex-end: flex-end value align the flex items to bottom of the container.

align-content

The align-content property used for align flex lines. To demonstrate the align-content property set flex-wrap property set to wrap.

stretch: stretch value stretches the flex lines to take up the remaining space.

center: center value show the flex lines in the center of the container.

flex-start: flex-start value show the flex lines in the start of the container.

flex-end: flex-end value show the flex lines in the end of the container.

space-around: space-around value show flex lines with space before, between and after the lines. It causes equal amount of space on the right and left of each lines.

space-between: space-between value align flex lines with equal space between the lines.

Now we know about flex item properties:

order: order property specify the order of the flex items. The order value must be a number, default value is 0.

<div class="flex-container">
  <div style="order: 3">1</div>
  <div style="order: 2">2</div>
  <div style="order: 4">3</div>
  <div style="order: 1">4</div>
</div

flex-grow: flex-grow property specifies how much flex item glow (increasing in size) relative to other flex items. The value must be a number, default value is 0.
The flex-grow property can be used to distribute space in proportion.

<div class="flex-container">
  <div style="flex-grow: 1">1</div>
  <div style="flex-grow: 1">2</div>
  <div style="flex-grow: 8">3</div>
</div>

flex-shrink: flex-shrink property specifies how much a flex item will shrink relative to other flex items. The value must be a number, default value is 1.
Do not let the second flex item shrink as much as the other flex items:

<div class="flex-container">
  <div>1</div>
  <div style="flex-shrink: 0">2</div>
  <div>3</div>
</div>

flex-basis: flex-basis values specifies the size of item in terms of space it leaves as available space.
Set the initial length of the second flex item to 20 pixels:

<div class="flex-container">
  <div>1</div>
  <div style="flex-basis: 20px">2</div>
  <div>3</div>
</div>

flex: flex property is a shorthand property for the flex-grow, flex-shrink, and flex-basis properties.
Make the first flex item growable (1), shrinkable (1), and with an initial length of 20 pixels:

<div class="flex-container">
  <div style="flex: 1 1 20px">1</div>
  <div>2</div>
</div>

align-self: align-self property specifies the alignment for selected item inside flex container.
Align the third flex item in the middle of the container:

<div class="flex-container">
  <div>1</div>
  <div>2</div>
  <div style="align-self: center">3</div>
  <div>4</div>
</div>

Here is a demo to play around about CSS Flex Box Module:

https://shafikshaon.github.io/css-flexbox/

Difference between stopPropagation and preventDefault?

Mohd. Shafikur Rahman — Fri, 17 Jan 2020 14:48:13 +0000

To know the difference between stopPropagation and preventDefault, we need to know about Event.

What is event?

The Event interface represents an event that takes place in the DOM tree to arrives at its target.

Event propagation has two phases:

Capturing: The browser checks to see if the element's outer-most ancestor <html> has an onclick event handler registered on it in the capturing phase, and runs it if so.

Then it moves on to the next element inside <html> and does the same thing, then the next one, and so on until it reaches the element that was actually clicked on.

The event starts from the window object down until it reaches the event.target.

Bubling: It's the opposite of capturing phase. The browser checks to see if the element that was actually clicked on has an onclick event handler registered on it in the bubbling phase, and runs it if so.

Then it moves on to the next immediate ancestor element and does the same thing, then the next one, and so on until it reaches the element.

The event bubbles up from the event.target element up until it reaches the window object.

preventDefault

The Event interface's preventDefault() method tells the user agent that if the event does not get explicitly handled, its default action should not be taken as it normally would be.

The event.preventDefault() method prevents the default behavior of an element. For example, it prevents <a href="/some-link/">Some link</a> elelment navigating.

Blocking default click handling

Toggling a checkbox is the default action of clicking on a checkbox. This example demonstrates how to prevent that from happening:

document.querySelector("#id-checkbox").addEventListener("click", function(event) {
         document.getElementById("output-box").innerHTML += "Sorry! <code>preventDefault()</code> won't let you check this!<br>";
         event.preventDefault();
}, false);

<p>Please click on the checkbox control.</p>

<form>
  <label for="id-checkbox">Checkbox:</label>
  <input type="checkbox" id="id-checkbox"/>
</form>

<div id="output-box"></div>

If you click on checkbox, then you will show something like this

Sorry!  preventDefault()  won't let you check this!

stopPropagation

The stopPropagation() method of the Event interface prevents further propagation of the current event in the capturing and bubbling phases.

By default, all event handlers are registered in the bubbling phase. So if you click on a HTML element, the click event bubbles from the clicked element outwards to the <html> element.

So, we need to use stopPropagation which makes it so that first handler is run but the event doesn't bubble any further up the chain, so no more handlers will be run.

$("#but").click(function (event) {
    event.stopPropagation()
})
// since propagation was stopped by the #but id element's click, this alert will never be seen!
$("#foo").click(function () {
    alert("parent click event fired!")
})

<script src="https://ajax.googleapis.com/ajax/libs/jquery/2.1.1/jquery.min.js"></script>
<div id="foo">
    <button id="but">button</button>
</div>

For IE9 and FF you can just use preventDefault & stopPropagation.

To support IE8 and lower replace stopPropagation with cancelBubble and replace preventDefault with returnValue

Kepp learning!

References:

Basic concepts of CSS Flexbox Layout

Mohd. Shafikur Rahman — Mon, 16 Dec 2019 14:08:22 +0000

The Flex Box Module usually referred to as flexbox, was designed as an one-dimensional layout model.
It make easier to design flexible responsive layout structure without using float or positioning.

Here is a demo to play around about CSS Flex Box Module: http://bit.ly/css-flexbox-layout

The flex container properties are:

flex-direction
flex-wrap
flex-flow
justify-content
align-items
align-content

flex-direction

The main axis defined by the flex-direction property. It defines in which direction the container want to stack the flex items.

The flex-direction has four possible values:

row
row-reverse
column
column-reverse

row: The row value stacks the flex item horizontally but from left to right.

row-reverse: The row value stacks the flex item horizontally but from right to left.

column: The column value stacks the flex item vertically but from top to bottom.

column-reverse: The column value stacks the flex item vertically but from bottom to top.

flex-wrap

Flex box is one dimensional model. So, it is possible to cause our flex items wrap into multiple lines. We should consider each line as new flex container.

flex-wrap property specifies whether items should wrap or not.
Wrap means cover or enclose (someone or something) in paper or soft material.

wrap: The wrap value specifies that the flex items will wrap if necessary.

nowrap: The wrap value specifies that the flex items will not wrap.

wrap-reverse: The wrap value specifies that the flex items will wrap if necessary but in reverse order.

flex-flow

flex-flow is shortland for both the flex-direction and flex-wrap properties

for example

.class {
    flex-flow: row wrap;
}

justify-content

The justify-content property is used to align the items on the main axis where direction determine by flex-direction.

flex-start: flex-start value align flex items at the start of the container.

flex-end: flex-end value align flex items at the end of the container.

center: center value align flex items at the center of the container.

space-around: space-around value align flex items with space before, between and after the lines. It causes equal amount of space on the right and left of each item.

space-between: space-between value align flex items with space between the lines.

space-evenly: space-evenly value align flex items with full-size space on either end.

align-items

The align-items property used for align flex items vertically.

stretch: stretch value stretch the flex items to fill the container. This is why flex items stretch to the height of the tallest one by default.

baseline: baseline value align the flex items by their flex container baselines.

center: center value align the flex items to center of the container.

flex-start: flex-start value align the flex items to top of the container.

flex-end: flex-end value align the flex items to bottom of the container.

align-content

The align-content property used for align flex lines. To demonstrate the align-content property set flex-wrap property set to wrap.

stretch: stretch value stretches the flex lines to take up the remaining space.

center: center value show the flex lines in the center of the container.

flex-start: flex-start value show the flex lines in the start of the container.

flex-end: flex-end value show the flex lines in the end of the container.

space-around: space-around value show flex lines with space before, between and after the lines. It causes equal amount of space on the right and left of each lines.

space-between: space-between value align flex lines with equal space between the lines.

Now we know about flex item properties:

order: order property specify the order of the flex items. The order value must be a number, default value is 0.

<div class="flex-container">
  <div style="order: 3">1</div>
  <div style="order: 2">2</div>
  <div style="order: 4">3</div>
  <div style="order: 1">4</div>
</div

<div class="flex-container">
  <div style="flex-grow: 1">1</div>
  <div style="flex-grow: 1">2</div>
  <div style="flex-grow: 8">3</div>
</div>

<div class="flex-container">
  <div>1</div>
  <div style="flex-shrink: 0">2</div>
  <div>3</div>
</div>

flex-basis: flex-basis values specifies the size of item in terms of space it leaves as available space.
Set the initial length of the second flex item to 20 pixels:

<div class="flex-container">
  <div>1</div>
  <div style="flex-basis: 20px">2</div>
  <div>3</div>
</div>

<div class="flex-container">
  <div style="flex: 1 1 20px">1</div>
  <div>2</div>
</div>

align-self: align-self property specifies the alignment for selected item inside flex container.
Align the third flex item in the middle of the container:

<div class="flex-container">
  <div>1</div>
  <div>2</div>
  <div style="align-self: center">3</div>
  <div>4</div>
</div>

Here is a demo to play around about CSS Flex Box Module: http://bit.ly/css-flexbox-layout

Originally posted on https://shafik.xyz/posts/basic-concepts-of-css-flexbox-layout/

Python: Generators and Iterators

Mohd. Shafikur Rahman — Sun, 15 Dec 2019 18:50:42 +0000

Iterators

Iterator is an object that can iterate (in loop) through object like lists, tuples, dicts, and sets.

Iterate over an object means traverse through the values of these object.

In python, iterator object implements in iterator protocol, which have __iter__() and __next__() (next in Python 2).

If you want any object to be an iterator. Then you must implement following method.

__iter__: __iter__ method is called the initializer of an iterator object. It return object that has

__next__ (nextin Python 2) method.
__next__: __next__ method return the next value for iterable. For loop implicitly call next on iterator object.

This method should raise a StopIteration to signal the end of the iteration.

iterable is an object which one iterate over. It generates an iterator when passed to iter() method.
iterator is an object, which is used to iterate over iterable object using next method.

Iterator have __next__ which return the next value of the iterable object.

name_tuple = ('John', 'Doe', 'Marry')
name_iterator = iter(name_tuple)

print(next(name_iterator))
print(next(name_iterator))
print(next(name_iterator))

Output

John
Doe
Marry

We create aiterator type that iterates from 0 to limit. For example, if we set limit 5 then it prints 1, 2, 3, 4, 5.

class CustomIterator:  
    def __init__(self, limit):  
        self.limit = limit  

      # called when iteration is initialized  
    def __iter__(self):  
        self.x = 0  
        return self  

      # move next value  
    def __next__(self):  
        x = self.x  
        if x > self.limit:  
            raise StopIteration  
        self.x = x + 1  
        return x  


for i in CustomIterator(5):  
    print(i)

Output

myclass = CustomIterator(5)
myitr = iter(myclass)

print(next(myitr))
print(next(myitr))
print(next(myitr))
print(next(myitr))
print(next(myitr))

Output

Some of built in iterations: lists, sets, dicts, and tuples.

Generators

Generator is a simpler way to create iterators. Iterators methods automatically handled by generators in python.

Generator returns an object(iterator) which we can iterate over (one value at a time).

We can create generator by defining a normal function with yield statement istead of return statement.

return statement terminate function entirely, yeild statement pause the functional and saving all states and later continues from there succescive calls.

def infinite_seq():
    num  = 0
    while True:
        yield num
        num += 1

x = infinite_seq()
print(x.__next__())
print(x.__next__())
print(x.__next__())
print(x.__next__())
print(x.__next__())

Output

yeild statement iterates where a value is sent back to the caller, but unlike return, you don't exit function afterward.

The state of generator function is remembered. When __next__ object called for generator object, the previously yielded variable num is incremented and then yeilded again.

Generator expressions creates an anonymous generator function.

Generator expression as follows

my_list = [1, 2, 3]
>>> (x**2 for x in my_list)
<generator object <genexpr> at 0x03DE6488>
>>>  sum((x**2 for x in my_list))
14

Generator Performance

>>> import sys
>>> num_list = [i**2 for i in range(100000)]
>>> sys.getsizeof(num_list)
412228
>>> num_list = (i**2 for i in range(100000))
>>> print(sys.getsizeof(num_list))
56

Generator is useful for reading large files. It's memory efficient.

Ubuntu Server setup on Virtual box and connect with SSH

Mohd. Shafikur Rahman — Tue, 10 Dec 2019 03:31:00 +0000

Here, I guide how to connect with ubuntu server installed on virtual box via SSH.

Install Virtual box and Ubuntu Server

Download virtual box from Oracle Virtualbox.

Install virtual box on your machine.

Download ubuntu server from ubuntu.

Then setup ubuntu server on virtual box.

After a successful setup, login to ubuntu server in virtual machine.

Setup SSH on ubuntu server

Then open terminal and install following packages:

sudo apt update
sudo apt install openssh-server
sudo apt install ssh

Once installation is completed, the SSH service should start automatically.
To verify the installation type following command:

sudo systemctl status ssh

You should see something like Active: active (running)

Now go to virtual box click on ubuntu instance and click on settings.
Then click on Network
Then choose Adapter 1 select Attached to to Bridge Adapter and Name to anyone.

Then run following command in ubuntu server.

sudo ufw disable

Login to server via SSH

Then login to server via ssh by following command

ssh your_username@your_server_ip_address

To see ip address type following command

ifconfig

In my case, ip address is 192.68.31.225 and username is test

ssh test@192.68.31.225

Optional

If you get timeout while connecting via ssh.

Stop your virtual machine.
Set the net of your virtual machine:
Click on settings and network:
Select NAT as connection and click on advanced.

Click on port forwarding:

Name: SSH
Protocol: TCP
Host IP: 127.0.0.1
Host Port: 2222
IP Guest: Empty
Port Guest: 22

Start your Virtual Machine:
When your virtual machine is started, open your terminal and try to connect:

ssh yourusername@127.0.0.1 -p 2222

Originally posted on https://shafik.xyz/posts/ubuntu-server-setup-on-virtual-box-and-connect-with-ssh/

User Registration With Email Verification

Mohd. Shafikur Rahman — Fri, 19 Jul 2019 13:38:25 +0000

Assume that You have a project named user_registration and it contains an app named accounts.

At first, we need to configure our email server. So, got to user_registration/setting.py and write the follwoing lines:

    EMAIL_BACKEND = 'django.core.mail.backends.smtp.EmailBackend'  
    MAILER_EMAIL_BACKEND = EMAIL_BACKEND  
    EMAIL_HOST = 'your_mail_server'  
    EMAIL_HOST_PASSWORD = 'your_password'  
    EMAIL_HOST_USER = 'your_email'  
    EMAIL_PORT = 465  
    EMAIL_USE_SSL = True  
    DEFAULT_FROM_EMAIL = EMAIL_HOST_USER

To test above configuration work, open up the terminal and navigate to user_registration project and run following command:

    $ python manage.py shell
    >>> from django.core.mail import send_mail
    >>> send_mail( 'Subject here', 'Here is the message.', 'me@shafikshaon.com', ['shafikshaon@gmail.com'], fail_silently=False, )

It will return 1 as status code.
Then check your mail, a mail will arrive.

Now create a form in accounts/templates/accounts/signup.html and write the following code:

    <!DOCTYPE html>  
    <html lang="en">  
    <head>  
        <title>Signup</title>  
        <meta charset="utf-8">  
        <meta name="viewport" content="width=device-width, initial-scale=1">  
        <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/4.1.3/css/bootstrap.min.css">  
    </head>  
    <body>  

    <div class="container" style="margin-top: 50px;">  
        <div class="row justify-content-center">  
            <div class="col-md-5 shadow-sm p-3 mb-5 bg-white rounded">  
                <h2>Signup</h2>  
                <form method="post">  
                    {% csrf_token %}  
                    <div class="form-group">  
                        <label for="first_name">First Name:</label>  
                        <input type="text" class="form-control" id="first_name" placeholder="Enter first name"  
      name="first_name">  
                    </div>  
                    <div class="form-group">  
                        <label for="last_name">Last Name:</label>  
                        <input type="text" class="form-control" id="last_name" placeholder="Enter last name"  
      name="last_name">  
                    </div>  
                    <div class="form-group">  
                        <label for="username">Username:</label>  
                        <input type="text" class="form-control" id="username" placeholder="Enter username" name="username">  
                    </div>  
                    <div class="form-group">  
                        <label for="email">Email:</label>  
                        <input type="email" class="form-control" id="email" placeholder="Enter email" name="email">  
                    </div>  
                    <div class="form-group">  
                        <label for="pwd1">Password:</label>  
                        <input type="password" class="form-control" id="pwd1" placeholder="Enter password" name="password1">  
                    </div>  
                    <div class="form-group">  
                        <label for="pwd2">Confirm Password:</label>  
                        <input type="password" class="form-control" id="pwd2" placeholder="Reenter password"  
      name="password2">  
                    </div>  
                    <button type="submit" class="btn btn-primary">Submit</button>  
                </form>  
            </div>  
        </div>  
    </div>  

    </body>  
    </html>

Now create a form in accounts/templates/accounts/acc_active_email.html and write the following code:

    {% autoescape off %}  
    Hi {{ user.username }},  
    Please click on the link to confirm your registration,  

    http://{{ domain }}{% url 'activate' uidb64=uid token=token %}  
    If you think, it's not you, then just ignore this email.  
    {% endautoescape %}

Now open up accounts/urls.py and write the following path:

    from django.urls import path  
    from accounts import views  

    urlpatterns = [  
        path('signup/', views.signup, name="signup"),  
        path('activate/<uidb64>/<token>/',views.activate, name='activate'),  
    ]

Now open up accounts/tokens.py and write the following code:

    from django.contrib.auth.tokens import PasswordResetTokenGenerator  
    from django.utils import six  


    class AccountActivationTokenGenerator(PasswordResetTokenGenerator):  
        def _make_hash_value(self, user, timestamp):  
            return (  
                six.text_type(user.pk) + six.text_type(timestamp) +  
                six.text_type(user.is_active)  
            )  

    account_activation_token = AccountActivationTokenGenerator()

Now open up accounts/forms.py and write the following code:

    from django.contrib.auth.forms import UserCreationForm  
    from django.contrib.auth.models import User  


    class SignUpForm(UserCreationForm):  
        class Meta:  
            model = User  
            fields = ('email', 'first_name', 'last_name', 'username')

Now open up accounts/views.py and write the following code:

    from django.http import HttpResponse  
    from django.shortcuts import render  
    from django.contrib.sites.shortcuts import get_current_site  
    from django.utils.encoding import force_bytes, force_text  
    from django.utils.http import urlsafe_base64_encode, urlsafe_base64_decode  
    from django.template.loader import render_to_string  

    from .forms import SignUpForm  
    from .tokens import account_activation_token  
    from django.contrib.auth.models import User  
    from django.core.mail import EmailMessage  


    def signup(request):  
        if request.method == 'GET':  
            return render(request, 'accounts/signup.html')  
        if request.method == 'POST':  
            form = SignUpForm(request.POST)  
            # print(form.errors.as_data())  
            if form.is_valid():  
                user = form.save(commit=False)  
                user.is_active = False  
                user.save()  
                current_site = get_current_site(request)  
                mail_subject = 'Activate your account.'  
                message = render_to_string('accounts/acc_active_email.html', {  
                    'user': user,  
                    'domain': current_site.domain,  
                    'uid': urlsafe_base64_encode(force_bytes(user.id)).decode(),  
                    'token': account_activation_token.make_token(user),  
                })  
                to_email = form.cleaned_data.get('email')  
                email = EmailMessage(  
                    mail_subject, message, to=[to_email]  
                )  
                email.send()  
                return HttpResponse('Please confirm your email address to complete the registration')  
         else:  
            form = SignUpForm()  
         return render(request, 'accounts/signup.html', {'form': form})  


    def activate(request, uidb64, token):  
        try:  
            uid = force_text(urlsafe_base64_decode(uidb64))  
            user = User.objects.get(id=uid)  
        except(TypeError, ValueError, OverflowError, User.DoesNotExist):  
            user = None  
        if user is not None and account_activation_token.check_token(user, token):  
            user.is_active = True  
            user.save()  
            return HttpResponse('Thank you for your email confirmation. Now you can login your account.')  
        else:  
            return HttpResponse('Activation link is invalid!')

When user signup, is_active set to False that means user set to inactive during signup process.
By render_to_string() method which data/ value send over conext. Here, data/ value send overaccounts/acc_active_email.html and user, domain, uid, and token data as context data.

Convert uid into text. Then, find user object by uid.
Now check account_activation_token.check_token(user, token) to check is this match with previously creaed token. If match the user.is_active set to True.

Interesting facts in JavaScript

Mohd. Shafikur Rahman — Thu, 18 Jul 2019 19:52:28 +0000

There is a great deal of amusing to be had when working in JavaScript. Notwithstanding for engineers that associate with it day by day, a few pieces of the language stay unexplored. I'm going to feature few things you may not think about JavaScript.

NaN is a number

NaN (Not a number) is being a number. Also, NaN is not equal to itself. Actually NaN not equal to anything. The only way to verify anything is NaN or not by isNaN().

> typeof(NaN)
"number"

> NaN === NaN
false

null is an object

null is an object. Sound odd! Right? But this is fact.

> typeof(null)
"object"

In case, null has no value. So, null should not instance of Object.

> null instanceof Object
false

undefined can be defined

undefined is not a reserved keyword in JavaScript. You can assign value to it. It doesn't through any syntax error. But, you can't assign value anywhere, it return undefined.

> var some_var;
undefined
> some_var == undefined
true
> undefined = 'i am undefined'

0.1 + 0.2 is not equal to 0.3

In JavaScript , 0.1 +0.2 == 0.3 return false. The fact is, how javascript store float number as binary.

> 0.1 + 0.2
0.30000000000000004
> 0.1 + 0.2 == 0.3
false

Math.max() smaller than Math.min()

The fact that Math.max() > Math.min() returns false sounds wrong, but it actually makes a lot of sense.

> Math.max() > Math.min()
false

If no argument pass via min() or max() then it return follwoing values.

> Math.max()
-Infinity
> Math.min()
Infinity

018 minus 045 equal to 3

In JavaScript, the prefix 0 will convert any number to octal. However, 8 is not used in octal, and any number containing an 8 will be silently converted to a regular decimal number.

> 018 - 045
-19

Therefore, 018 — 017 is in fact equivalent to the decimal expression 18 — 37, because 045 is octal but 018 is decimal.

Functions can execute itself

Just create a function and immediately call it as we call other functions, with () syntax

> (function()  { console.log('I am self executing');  })();
I am self executing

Parenthesis position matter

The return statement “does not see” that it has something to return so it returns nothing. Actually, JavaScript put ; after return.

> function foo() {
   return
   {
      foo: 'bar'
   }
}
> foo(); 
undefined

> function foo() {
   return {
      foo: 'bar'
   }
}
> foo(); 
{foo: "bar"}

Missing parameter default value

In JavaScript, you can set a parameter default value in the following way.

> function missingParamerCheck(name, age){
    var name = name || 'John Doe'
    var age = age
    console.log(name)
    console.log(age)
}
> missingParamerCheck('', 23)
John Doe
23
> missingParamerCheck('Mike', 18)
Mike
18

Doesn't have integer data type

In JavaScript, there is not int(integer) data type. All numbers are Number type. Actually it store float value for int number in memory level.

sort() function automatic type conversion

The sort() function automatic convert value to string, that why something weired being happened.

> [1,5,20,10].sort()
(4) [1, 10, 20, 5]

But, it can be fix by comparing.

> [1,5,20,10].sort(function(a, b){return a - b});
(4) [1, 10, 20, 5]

Sum of Arrays and Objects

> !+[]+!![]+!![]+!![]+!![]+!![]+!![]+!![]+!![]
9
> {} + []
0
> [] + {}
"[object Object]"
> [] + []
""
> {} + {}
"[object Object][object Object]"
> {} + [] == [] + {}
true

Ideally, you discovered some new information or possibly showed signs of improvement comprehension of what is new with these JavaScript pearls. What other unexplored/unprecedented JavaScript highlights do you know? Share them in the comments.
This post is also available in here