DEV Community: Surya Teja

Automated testing for staging Apps

Surya Teja — Tue, 16 Nov 2021 05:48:48 +0000

My Workflow

A blog source for entire process: https://niteo.co/blog/staging-like-its-2020
A GitHub Action: https://github.com/marketplace/actions/heroku-review-app-deployment-status

Submission Category:

Testing of staging Apps

Yaml File or Link to Code

https://github.com/marketplace/actions/heroku-review-app-deployment-status

Additional Resources / Info

Everything is covered in the above mentioned blog post.

Automated testing for staging Apps

Surya Teja — Tue, 16 Nov 2021 05:41:30 +0000

I have developed a process to update the pull request status based on staging app deployment using Github Actions and Heroku API.

Entire journey and workflow can be found here: https://niteo.co/blog/staging-like-its-2020

The journey consists of:

A simple custom GitHub Action for small & medium projects. It takes care of all the burden.
A bit complex process for larger projects which can be helpful in reducing unnecessary costs.

Python Dictionaries and their operations

Surya Teja — Sun, 07 Jul 2019 15:55:40 +0000

The whole point of programming concludes to one inevitable out-coming i.e., we write instructions to manipulate/control DATA.

Python provides many data structures to store data, Dictionary is one among them which store the data as a key-value pair. According to official documentation, there are many definitions but, I find the below definition easy to understand.

Dictionaries are a type of data structures, which are indexed by keys. It can also be defined as an unordered set of key: value pairs. Immutable(not dynamic) types like strings and numbers can always be keys.*

Basic operations on Dictionaries

As mentioned in the first point, python provides no.of ways to manipulate/control data. Below mentioned are a few basic methods used in dictionaries.

Loops on Dictionaries

On dictionaries, python provides three basic methods to loop through the items
of the dictionary.

dict.items()
dict.keys()
dict.values()

Other methods on Dictionaries

Python also provides methods which can be really helpful in reducing the size of
code by using reusable modules.

Comparing dictionaries

The compare method cmp() is used to compare length of two dictionaries. The method returns 0 if both dictionaries are equal, 1 if dic1 > dict2 and -1 if dict1 < dict2.

    >>> Boys = {'sunny': 26,'Chary':18,'Robin':25}
    >>> Girls = {'pari':22} 
    >>> print(cmp(Girls, Boys))
    -1

get(key[, default]) Method

Return the value for key if the key is in the dictionary, else default. If
default is not given, it defaults to None so that this method never raises a KeyError.

get() is a very useful function as we don’t need to loop through the dictionary items to check whether a certain key is present in the dictionary. For nested dictionaries, it reduces a lot of pain to loop and checks as well as it makes the code more readable.

    # dict.get( key, default )
    >>> Boys.get('sunny')
    26
    >>> Boys.get('surya', 'Not Found')
    'Not Found'
    >>> Boys.get('surya')
    None

Merging two dictionaries to one dictionary

There are many methods to merge two dictionaries into a single dictionary. Using the multiple arguments ‘**’ operator we can achieve this.

      >>> x = {'a': 1, 'b': 2}
      >>> y = {'b': 3, 'c': 4}
      >>> z = {**x, **y}
          {'a': 1, 'b': 3, 'c': 4}

Note: This method is applicable only in python 3.5. When there are conflicting
keys, the interpreter chooses the latest value.

json.dumps()

The standard string representation for dictionaries is hard to read. Using the json.dumps we can make the output as JSON with more readability.

      # The standard string representation for dictionary:
      >>> obj = {'a': 'asdasd', 'b':'rgggsd', 'c':'asdsd'}
      >>> obj
          {'a': 'asdasd', 'b':'rgggsd', 'c':'asdsd'}
      >>> import json
      >>> print(json.dumps(obj, indent=4))
          {
             "c": "asdsd",
             "b": 42,
             "a": "asdasd"
          }

json.dumps will accept multiple parameters. As shown above, we can give indent level and we can supply other parameters like sort_keys.

    >>> print(json.dumps(obj, indent=4, sort_keys=True))
    {    
       "a": "asdasd",
       "b": "rgggsd",
       "c": "asdsd"
    }

json.loads()

loads() performs the vice-versa of the dumps. It takes the JSON input and converts it back to the dictionary.

      Jobj = {
         "a": "asdasd",
         "b":"rgggsd",
         "c":"asdsd"
      }
      >>> import json
      >>> print(json.loads(jobj))
      {'a': 'asdasd', 'b':'rgggsd', 'c':'asdsd'}

Dictionary as switch/case statements

Python doesn’t have switch/case statements as default. But as python has
first-class functions, they can be used to emulate switch/case statements.
Following is an example of implementation of switch/case statement using the dictionary.

Conclusion:

Python dictionaries very powerful when compared to other data types like lists,
tuples and sets. In other languages, these are also called as “associative
memories” or “associative arrays”. There are lot more properties and methods for
dictionaries which are used as per requirement.

Sources:

Multi-Threading in Python

Surya Teja — Sun, 07 Jul 2019 15:04:28 +0000

Multi-Threading in Python using the ‘Activity’ Application

Threads in the computer world are described as simple executors (which execute the given instructions). These are lightweight and used to perform tasks asynchronously.

Eg: Different tabs in browser act as different threads

In general, At the OS level, threads have their own resources but at an abstract level, they use the same code, data and files.

Source: https://www.studytonight.com

Multithreading in activity application:

Activity is an application developed by my good friend Arijit. It performs bulk server command execution and validations over ssh asynchronously.

It was developed in python 3.6. The core concept of this module is that, it uses multi-threading to perform command execution in parallel on all the given servers because of which it consumes less time and performs tasks in an organised and efficient way.

To understand multithreading, Let's dive into some code which is part of
activity app, which performs ping check on all the hosts asynchronously.

The above-defined function is a callable function, which performs the required activity.

To understand this, we should know a brief background of how parallel execution works in python.

concurrent.futures: Launching parallel tasks

The concurrent.futures module provides a high-level interface for asynchronously executing callables which is an In-built module.

In our program, _ping_single(args): is a executing callable function

The asynchronous execution can be performed using threads and processes. Sub-function for threads in concurrent.futures is ThreadPoolExecutor and processes is ProcessPoolExecutor.

To import required module

from concurrent.futures import ThreadPoolExecutor, as_completed

Once imported,

with ThreadPoolExecutor(max_workers=PARALLEL_LIMIT) as pool:

The above line is used to initialize the threads. It is initialized with with so that when execution of a threads complete, the resources of the thread are freed and then shutdown the Executor.

The execution starts once the callable function is submitted along with
arguments as shown above. In the above line hosts, as well as required arguments are submitted along with callable function.

We can use pool.map to submit the tasks to threads, The difference would be seen in while capturing the output.

    for f in as_completed(futures):
        output.append(f.result())

Once the execution of the thread is started, we can capture the output using as_completed or the wait function.

The as_completed() function takes an iterable of Future objects and starts yielding values as soon as the futures (threads) start resolving. The main difference between the aforementioned map method with as_completed is that map returns the results in the order in which we pass the iterables. That is the first result from the map method is the result of the first item. On the other hand, the first result from the as_completed function is from whichever future completed first.

The wait() function would return a named tuple which contains two set — one set contains the futures which completed (either got result or exception) and the other set containing the ones which didn’t complete.

Note: If the threads are not configured properly, there will be performance issues like deadlock, memory leaks.

Conclusion:

Multithreading if configured properly, is an efficient and organized way to reduce the time taken for execution as well as improve the performance of the program.

For more explanation on multithreading please refer the below article:
https://medium.com/@bfortuner/python-multithreading-vs-multiprocessing-73072ce5600b

Source:

Raspberry pi’s RPI GPIO Module of python

Surya Teja — Sun, 07 Jul 2019 14:31:31 +0000

Basic setup and working with raspberry pi’s RPI GPIO Module of python

The first post is about basic setup and working with raspberry pi’s RPI GPIO module of python.

Brief intro to Raspberry Pi:

Raspberry Pi is a credit card-sized computer designed for education. The first model was launched in 2012 which was basically used for education

Source: https://www.raspberrypi.org

Features:

Small size
Low-cost Device
Complete linux computer
Low power consumption

GPIO pins:

There are 40 powerful General Purpose Input/Output(GPIO) pins which are part of
core components of the Raspberry pi. There are two 5V pins and two 3.3v, ground
pins and general purpose pins.

Source:
https://www.raspberrypi.org/documentation/usage/gpio/images/gpio-numbers-pi2.png

RPI GPIO Module:

Python can be used to write applications in Raspberry pi. It comes with a powerful package manager called PIP which can be used to install third-party packages.

RPI GPIO Module is a third-party package which can be used to configure and control the GPIO pins.

The basic setup is pretty easy. After setup, we define whether a pin’s value should be high or low.

Basic setup

Make sure that python>2.7 is installed in the raspberry Pi.
Install the module using PIP

    sudo pip install RPi.GPIO

Note: If you face any problem in installing the package install python-dev and try again

Import the modules to script
Set the pin numbering schemes (explained below)

    GPIO.setmode(GPIO.BCM)

Configure the required pins

    GPIO.setup(18, GPIO.OUT)

    # PIN 18(BCM) is now used as output

Pin numbering schemes:

There are two types of pin numbering schemes

GPIO.BOARD: Board numbering scheme

The option specifies that you are referring to the pins by the number of the pin the plug i.e the numbers printed on the board.

GPIO.BCM: Broadcom chip-specific pin numbers.

The option means that you are referring to the pins by the “Broadcom SOC channel” number.

Once the basic setup is completed, Based on the requirement we set the pins to
high or low

    # Import modules
    import RPi.GPIO as GPIO
    import time
    Import atexit

    #PIN setup
    GPIO.setmode(GPIO.BCM)
    GPIO.setup(4, GPIO.OUTPUT)

    # Loop to send HIGH and LOW to the mentioned pin
    While True:
        GPIO.output(4,GPIO.HIGH)
        time.sleep(1)
        GPIO.output(4,GPIO.LOW)
        time.sleep(1)

    #Garabage Cleaner
    def clean_up():
        GPIO.cleanup()

    atexit.register(clean_up)

The above program sends a HIGH signal to 4th pin(BCM) for one second and LOW signal for the next one second continuously until the program is exited. To verify the output of the process we can connect LED and test it.

The first 5 lines of the program are a basic setup and the following loop
continuously sends a HIGH and LOW signal to the 4th pin until the program is
existed.

GPIO.cleanup() function is used as a garbage cleaner and is recommended to use in
every program which uses GPIO pins of the raspberry pi.

atexit.register() function is used to trigger a function before exiting.

Based on the requirement of the application, we can set the pin in different
modes.

Eg: To generate an Analog Output

Execution:

Once the program is completed, It can run by providing the following command into
the terminal

sudo python <program_name>.py

To exit the program

press ctrl+c

Sources:

Rpi.GPIO Module:
https://pypi.python.org/pypi/RPi.GPIO

Raspberry Pi Documentation:
https://www.raspberrypi.org/documentation