DEV Community: Manisha Naidu

What is a Jenkinsfile? how to write one?

Manisha Naidu — Tue, 19 Jul 2022 07:40:50 +0000

If you know about Jenkins then I am sure you would know about Jenkins pipeline. It is a suite of tasks which are linked to each other and are executed to achieve continuous integration and delivery.

And Jenkinsfile is the heart of this pipeline. It provides Jenkins pipeline a definition, with a text based file which contains everything that Jenkins pipeline needs to know for its execution.

Jenkinsfile is not the only way to define a pipeline, you can also do it in UI. However, it is a best practice to do it using Jenkinsfile and check-in to your SCM.

Here is the most basic pipeline to build, test and deploy a software,

pipeline{
    Agent any
    stages{
        stage("build") {
            steps{
                echo"This is the build step"
            }
        }
        stage("test"){
            steps{
                echo"This is the test step"
            }
        }
        stage("deploy"){
            steps{
                echo"This is the deployment stage"
            }
        }
    }
}

Mentioning agent is crucial, as it tells Jenkins to provide a host and workspace for execution.

Steps is where you will mention all the commands that needs to be executed for each stage.

As I said, this is a very basic pipeline and we can do much more than this in Jenkinsfile, such as, using environment variables to add conditions for stages/steps or even create your own variables.

We can also run the stages in parallel instead of sequential to increase the performance of pipeline.

I will be writing an in-depth article in a few days, which will cover all the essentials of Jenkinsfile and how we can use it to achieve CI-CD.

Hope this gave you an overview to Jenkinsfile.

Have a great day 🙂

If you want to support my work,

https://www.buymeacoffee.com/manishanaidu

Why Do We Need Design Principles?

Manisha Naidu — Tue, 14 Sep 2021 09:59:01 +0000

Are design principles really required while developing a software? Is it worth spending time on it instead of just focusing on making the code work?

Today I would like to talk about design principles, what are these? why do we need it? and some examples of them. Because I believe it is highly underrated while building a software or even during code reviews.

Now you might think that we have built some amazing programs, which works flawlessly in production and we did not waste time on following any design principles.

And you are probably right, but guess what? such programs are not going to last long. Most of the software we build will be used for a long period of time and as the days goes by, the same software is going to evolve with new features every week, if not every day. Moreover, there are going to be multiple developers working on the same software and everyone has different styles of coding. So if the code is not maintainable, even a small change in code will have a ripple-effect elsewhere and it will be very hard to reuse any of it and fixing even a small bug will be expensive.

So how do we make sure that our software is scalable and less prone to bugs in long run?

Well, first and foremost step is to make our code robust, reusable and chances of failure to be reduced. This can be done by giving importance to the quality of code and not just to functionality or output of the code. However, at the speed at which new version software are being released, it becomes a tad bit challenging to spend time on code quality.

Since not every developer needs to break their head to find new ways or standards which would best fit their software, there are already some treasured design principles out their, which are mostly suited for all types of software.

Be noted that these design principles are not the actual implementation for the system, instead, it provides you a way to achieve quality code for your implementation which will help in the long run for,

Flexibility
Reusability
Cost efficient
And most important, easily changeable, because the only constant in the tech industry is, you know what I'm going to say next, the CHANGE.

Now, there are multiple principles which you can choose from such as,

DRY - Don't repeat yourself

KISS - Keep it simple, stupid

YAGNI - You ain't gonna need it

We will be discussing SOLID principle, which is my favorite and covers almost everything you would ever need to use. Here is the description of the principle with some examples,

S: Single responsibility ~ "A class should have only one reason to change. If there is any other reason create another class."

While the definition is quite clear, it means that a class should have only one job. This principle focuses on coupling and cohesion. The classes should be very loosely coupled, whereas the relations inside a class should be tightly coupled.

Eg. class Order, should deal with only the orders of an e-commerce site and not include billing functionality in it.

O: Open-Closed Principle ~ "Entities like classes, modules, functions or objects should be open for extension but closed for modification."

Once a class/function is built, it should only be "open" for fixing bugs or adding new functionality and "closed" for anything else.

L: Liskov substitution states that "Derived class must be able to substitute its base class"

When we have a class inherited from a base class, the inherited class can be used instead of its base class by any pointer or reference.

Eg. If Customer class is derived from User class, then wherever we use User class, we should be able to use Customer class too.

I: Interface segregation "Clients should not be forced to depend on methods in interfaces that they don’t use."

This principle states that the client should not have to have a functionality in it which it does not use, but for some reason it still needs to be present in it.

Eg. class Person is an abstract class which has methods such as, show_subjects, enroll_lectures, get_job, get_salary. If a class Employee is implementing the class Person, it will have to implement show_subjects and enroll_lectures even if it does not need it. Hence, class Person needs to be separated into two interfaces , one will have methods useful to Students and other to Employees.

D: Dependency inversion states that "the high-level modules should not depend on low-level modules, it should rather depend on abstraction"

This helps in reducing coupling between classes by adding an abstraction layer.

Eg. class Teacher defining the methods of adding new lectures into the courses,

class Teacher():
    def __init__(self):
        self.english = []
        self.spanish = []
    def addEnglishStd(self,engStd):
        self.english.append(engstd)

  def addSpanishStd(self,spanStd):
        self.spanish.append(spanStd)
class English(object):
    def __init__(self):
        print "english student enrolled"

class Spanish(object):
    def __init__(self):
        print "spanish student enrolled"

With this approach, if another language to be added under a teacher, we have to make changes to complete Teacher class, which is against dependency inversion principle. To avoid this we can create a class Language,

class Language():
    def write():
        pass
    def speak():
        pass
class Teacher():
    def __init__(self):
        self.languages = []
    def addLanguage(self, language):
        self.languages.append(language)
class English(Language):
  def speak(self):
        print "Student to learn speaking english"
    def write(self):
            print "Student to learn writing english"

class Spanish(object):
  def speak(self):
        print "Student to learn speaking spanish"
    def write(self):
            print "Student to learn writing spanish"

Now if any new language can be added to Teacher without having to change the Teacher class.

Python provides a module called “abc” for Abstract Base Class, which can be explored for creating such interfaces.

Hope you gained some insights on design principles and would like to spend some time on it in your upcoming project, more articles on system design coming soon in this space.

If you want to support my work, https://www.buymeacoffee.com/manishanaidu

Python/Django Developer Interview Questions and Answer (#1)

Manisha Naidu — Mon, 03 May 2021 10:34:37 +0000

Most of the software engineers are intimidated by technical rounds and I have recently started to give interviews. So, this time I am planning to write up all the questions I get asked in my technical rounds. Hope this will be helpful to some of you to get an idea on what type of questions are being asked. Use this series as a guide to prepare and ace that interview.

Some questions might be specific to Python and Django but most of them will be applicable to any software engineering role and web application developers.

This interview was for a Python developer role and they were looking for a candidate proficient in Django too, hence you will be seeing many questions related to Django architecture and design as well.

1. Let’s work on a recursive implementation of Fibonacci series.

passed "n", return the fibonacci value at the nth position.

Solution: Fibonacci series are created in one of the two ways, iterative and recursive. Interviewer was looking for a recursive solution only.

def fibonacci(n):
    if n == 0 or n == 1:
      return n
    return fibonacci(n-1) + fibonacci(n-2)

This solution worked fine for smaller numbers,
n=10, took 0:00:00.001004sec
n=35, took 0:00:05.157989sec which is too long.
So next I was asked to optimize this.

Ideal Solution: Using Dynamic programming, I am storing each value found in a dictionary, so that the next it looks for fibonacci of that number it can just look-up from "value_dict"

def fibonacci(n, value_dict):
    # the first two values of fibonacci is always 0,1
    if n == 0 or n == 1:
        return n
    # if fibonacci(n) is already calculated then return it from dict 
    if value_dict.get(n):
        return value_dict[n]
    #else store it in dict and return next time it is called for.
    else:
        value_dict[n] = fibonacci(n-2,value_dict) + fibonacci(n- 
        1,value_dict)
        return value_dict[n]

print(fibonacci(100, {}))

2. What are decorators? Write a decorator that logs the functions arguments.

Solution:
Decorator is any function that takes in another function and adds functionality to it without manipulating the function itself. It does this by using a wrapper function.

def decorator(func):
    def wrapper(*args):
        print("Logging the parameter of ", func, " is ", args)
        return func
    return wrapper

@decorator
def operation(x, y):
    return x+y

operation(5,20)

@decorator is just a syntactic sugar, it is equivalent to
res = decorator(operation(5,20))

3. What GIL (global interpreter lock)? How does it allow multi-threading in python?

Solution: GIL is a lock which makes sure python interpreter is held by only one thread at a time. That means only one thread can be in a execution status at any point of time. And this is why python is a single threaded programming language. This was introduced to solve the reference count problem which is used by python for garbage collection.

Then how is multi-threading module working in python, you may ask!
Well, this global interpreter lock applies only for CPU bound activities. So if there is any code which affects or uses CPU it automatically starts acting as a single threaded. But all the general programs can still work in a multi-threaded fashion.

4. What is WSGI and UWSGI?

Solution:
Web server gateway interface, as the name suggests it is a interface between the application itself and web server.
Django provides runserver for development server and for debugging, but for production we use WSGI in combination of any web servers like nginx, apache, or even uwsgi server.
wsgi and uwsgi are both protocols, and these work with web servers to serve good performance application in production.

So when client makes a request to django application, the request is read by nginx or any web server. Then nginx passes this request to the uwsgi service which intern passes it to concerned django app.

5. Can we write a custom queryset in dango? How?

Solution:
Yes, we can create a custom queryset by customizing Managers. A Manager is the interface through which database query operations are provided to Django models. Each model contains atleast one Manager.
We can override Managers base queryset by override Manager.get_queryset() method.

class Employees(models.Model):
      name = models.CharField(max_length=50)
      location = models.CharField(max_length=100)

On running, Employees.objects.all() it will return all the employees in database.
Now, if we want to customize this to return only the employees from Toronto location then below can be done using Manager.

class TorontoEmployeesManager(models.Manager):
      def get_queryset(self):
          return super().get_queryset().filter(location="Toronto")

class Employees(models.Model):
      name = models.CharField(max_length=50)
      location = models.CharField(max_length=100)

      objects = models.Manager()
      toronto_objects = TorontoEmployeesManager()

Now, Employees.toronto_objects.all() will return only the employees from Toronto location.

6. arr1 = list(range(10)); arr2 = arr1; arr3 = arr1[:]

Is arr2 == arr3 ?

Solution:
No, arr2 = arr1 creates a new reference of arr1 and assigns it to arr2. whereas, arr3 = arr1[:] copies the content of arr1 and assigns it to new variable called arr3.

arr1 = [1,2,3,4]
arr2 = arr1
arr3 = arr1[:]
arr1.append(5)
print(arr2) # [1,2,3,4,5]
print(arr3) # [1,2,3,4]

7. How to hide a class variable from getting accessed outside the class?

class Sample:
  def __init__(self, a, b):
    self.a = a
    self.b = b

a and b should be hidden from outside of class.

Solution:

class Sample:
  def __init__(self, a, b):
    self.__a = a # using double underscore hides class variable
    self.__b = b

8. What are middlewares in Django? How do you create custom middleware.

Answer:
Middleware is a low-level plugin that can be used to hook into Django's request/response cycle. There are already middlewares available by default when a django project is created and it can found in settings.py.
We can create a custom middleware and include it in the list of middleware with the existing ones.

To create a custom middleware following structure needs to be followed:

# Middleware class should always consist __init__() and __call__()
class MyCustomMiddleware():
      def __init__(self, get_response):
          self.get_response = get_response #get_response is the view which will be called after this middleware/ or its the next middleware in the list.

      def __call__(self, request):
          # write the code to be executed before calling view here

          response = self.get_response(request) #pass the request to view and get the response back, which will be returned.

          # write the code to be executed after calling view here.
          return response

Add this middleware into the "Middleware" section in settings.py. Also make sure the order of the middleware is correct. Because it is called from top to down while processing requests and then bottom to up while returning response.

Hope this was helpful!

If you want to support my work, https://www.buymeacoffee.com/manishanaidu

Most important Python concepts for beginners to intermediate

Manisha Naidu — Tue, 12 Jan 2021 10:58:22 +0000

If you have 0-3 years of experience working in tech and you are going to apply for python developer role or if you are starting with python to build a project, then you must know some crucial concepts of the language.
Below you will find the most important and frequently asked topics in python interviews.
There are some amazing links to learn and master python at the end of this blog.

Different data structures in python
- List
- Tuple
- Sets
- Dictionaries
Valid types of keys in dictionary and their properties.
- Iterating a dict
Zip function
Difference between arrays and list in python.
String manipulation and slicing operation
Regex
List comprehension
Lambda/anonymous functions
Filter, map, reduce functions
Generators
Decorators
Exception handling
Files input and output using with
Deep and shallow copy
Memory management in python
Modules in python
Local vs global variables
_init_
*args, **kwargs
Multithreading

Python’s official documentation is very precise and coherent, so going through that would be helpful to understand above concepts. But if you are someone who understands better visually than reading, then here are some useful links,

For absolute beginners into programming/python:
python for beginners

Important data structures in python:

Data structures in python video 2. Documentation of DS

Comprehensions: Python Comprehensions
Generators in python: Python generators
Decorators: [Python decorators]https://youtu.be/FsAPt_9Bf3U
Multiprocessing in python: multiprocessing
File operations: Input/Output operations

The definitions of these concepts can be quite clear; however, you will need to understand them exceedingly to truly use them in your projects. The interviewers also ask tricky questions which can be answered only if you know the concepts thoroughly.

Connect to ActiveMQ using python STOMP

Manisha Naidu — Tue, 12 Jan 2021 09:16:44 +0000

ActiveMQ is an open source message-oriented middleware which is used to transfer messages between multiple applications. It consists of a queue which will hold in the messages and will only transfer when the receiver is available, hence we can be rest assured that the messages won’t be lost if the server is down. The applications could be entirely different and hosted on heterogenous platforms, ActiveMQ can be used to communicate between such systems as well.

For example, large applications built in microservices architecture and written in disparate programming languages, where communication is often but an offline service should not cause any loss of messages.

Working of ActiveMQ:
There are two endpoints in ActiveMQ,

Producers – who send the messages to the message broker.
Consumers – who read these messages one at a time.

ActiveMQ stores the messages in the form of queue or topic.

Queue: A message is sent to a single consumer and if the consumer is unavailable then it is held by activemq until the consumer is available to receive the message.
Topic: A message copy is sent to all of the subscribers who are interested in a particular topic. Unavailable consumers will not be receiving the past messages.

Now let us see how to create this consumer code and subscribe to an ActiveMQ topic in Python.
Python provides a client library called stomp.py for connecting with any type of message brokers using STOMP protocol.

Step1: install the latest version of stomp available.
pip install stomp.py

Step2: The consumer script.

import stomp
#Establish a connection
con = stomp.StompConnection([(‘Host’,port)])
#listener class to be instantiated.
listener = MsgListener()
con.set_listener(‘name_of_listener’, listener)
#wait will ensure it waits till connection is established and acknowledged.
con.connect(‘usernanme’, ‘password’, wait=True)
#subscribe to a particular topic or queue by giving the path and headers if required by the server.
con.subscribe(‘topic/path’, headers={})

Step3: Listener class for your processing.

Class MsgListener(stomp.ConnectionListener):
    def __init__(self):
        # to keep the count of messages received
        self.msg_recieved = 0 

    def on_error(self, message):
        logger.error(‘received an error “%s”’ %message)

    def on_message(self, message):
        message = json.loads(message)
        self.msg_received +=1
        # add your logic based on the message received here

If the con.subscribe is being done to a topic, then a published message will go to all the subscribers who have active subscription at the time of message published.
Whereas, if it is a queue only one consumer will receive one message. Messages are load balanced among the available consumers.