DEV Community: Alwoch Sophia

Understanding the impact of Python variable assignment technique using the Fibonacci sequence

Alwoch Sophia — Fri, 26 May 2023 07:59:40 +0000

Python offers two ways to assign variables; single and multiple variable assignment. The choice of which variable assignment technique you use sometimes has a tremendous effect on your code's accuracy.

This article will explain the two variable assignment methods and will use the Fibonacci sequence to illustrate the variation in results caused by each of them.

Prerequisites

To follow along, you should have:

A code editor such as VScode installed on your computer or online IDE such as replit
Python installed if you're using a code editor locally on your machine.

single variable assignment

Under single variable assignment, only a single value is assigned to a single variable as follows:

x=5

If under the first x variable assignment, you reinitialised x with the value 10 and printed it, the previous value would be overwritten and you'd have 10 as the new value of x as shown below:

multiple variable assignment

Under multiple variable assignment, multiple variables can reference a single value as shown below:

variable assignment in Python follows the right-to-left rule and what happens in this case is that the value 10 is first assigned to the variable r, then r assigned to q and finally, q gets assigned to p.

Also under multiple variable assignment, multiple values can be assigned to multiple variables on a single line as shown below:

In the above example, a tuple of (5,10,15) is created and iterated over to assign the values to a,b and c

Now you must be thinking "Oh I could just choose the single variable assignment and stick with it" -but sometimes it's not one size fits all. Depending on the problem, a change in variable assignment may be required to attain the desired level of accuracy.

The Fibonacci Sequence

In mathematics, the Fibonacci Sequence is a sequence in which each number is the sum of the two preceding ones. Numbers that are part of the Fibonacci sequence are known as Fibonacci numbers. The sequence commonly starts from 0 and 1, although some authors start the sequence from 1 and 1 or sometimes (as did Fibonacci) from 1 and 2. Starting from 0 and 1, the first few values in the sequence are:

0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144.

Wikipedia

To break it down, there are two initial numbers in the sequence; 0 and 1. The next Fibonacci number is the sum of 0 and 1 which is 1. The sequence now consists of 0,1,1. The next number would be a sum of the last two numbers in the sequence - 1 and 1 to make it 0,1,1,2 and so on....

In this example, the goal is to write a function with a parameter n -that will generate Fibonacci numbers using n as the limit. The steps to achieve this are broken down as follows:

create variables x and y to store the initial values 0 and 1 respectively
create an empty array for the Fibonacci numbers
loop through the limit to add the Fibonacci numbers
in the loop, set x to the value y and y to the sum of x and y
return the array of Fibonacci numbers

The above pseudo-code now becomes:

def generate_fibonacci(n):
  x= 0
  y=1
  fibs = []
  while x < n:
    fibs.append(x)
    x = y
    y = x + y
  return fibs

print(generate_fibonacci(100))

When you run this code, you should have the following result in your terminal

Just to be sure the function is returning accurate results, add a unit test to test its output

import unittest

class FibonacciGeneratorTest(unittest.TestCase):

  def test_generate_fibonacci(self):
    # Test the Fibonacci generator for various inputs
    test_cases = [(0, []), (1, [0]), (2, [0, 1, 1]),
                  (10, [0, 1, 1, 2, 3, 5, 8]),
                  (100, [0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89])]

    for n, expected_sequence in test_cases:
      with self.subTest(n=n):
        self.assertEqual(generate_fibonacci(n), expected_sequence)

if __name__ == ' __main__':
  unittest.main()

In the above code, we import the unittest module and write a test for the generate_fibonacci function. An array of test cases is added with the limit being the first item in each tuple and the expected Fibonacci sequence as the second item. We then loop over the test cases and assert that the value returned from generate_fibonacci(n) is equal to the expected Fibonacci sequence.

Run the test and you should have a result in the teminal

Well, looks like the test failed! An indicator that the generate_fibonacci function is not working as expected and will need to be refactored to provide the desired result.

When you critically look at the sequence from the generate_fibonacci function, you can observe that the proceeding numbers are multiples of the preceding numbers -implying the problem could be in the while loop.

def generate_fibonacci(n):
#.....
  while x < n:
    fibs.append(x)
    x = y
    y = x + y
  #......

from the while loop, after appending the initial x value (0) to the fibs array, x is set to the value of y (1); y becomes the sum of 1 and 1 -adding 1 to the sequence. Thereafter, subsequent iterations generate multiples of the preceding number. This is not what we're trying to achieve!

To solve the above problem, refactor the code using multiple variable assignment:

def generate_fibonacci(n):
  x = 0
  y = 1
  fibs = []
  while x < n:
    fibs.append(x)
    x, y = y, x + y
  return fibs

print(generate_fibonacci(100))

Run the code and you should have a result in your terminal

Now you have an accurate list of Fibonacci numbers of the specified limit and the test passed too!

So what changed? Well, using multiple variable assignment made it possible to assign new values to x and y based on their existing values. In the previous implementation, reassigning the value of x separately from y meant y would become the sum of the new value of x and y -providing inaccurate results.

Besides reassigning variables based on their existing values, the multiple variable assignment technique is also commonly used when swapping values between variables as shown below:

Conclusion

If you made it this far you've understood Python variable assignment methods and how each of the methods is likely to impact your code's accuracy.

How to serialize responses in nestjs

Alwoch Sophia — Mon, 01 May 2023 08:56:30 +0000

When writing APIs, you'll most likely have to return responses to the client. For example, when writing a sign-up API, you may be required to return the created user object to the client.

An important step to take in development is ensuring that only the required amount of data is returned for performance reasons -and that It does not include sensitive information such as passwords that could compromise your users' accounts. This can be achieved by serializing responses before they are returned..

In this tutorial, you'll learn how serialization works in nestjs and how to serialize your responses in nestjs

prerequisites

To follow along seamlessly, this article assumes you have:

some experience building REST APIS with nodejs and nestjs
An understanding of nestjs project structure and terminologies such as decorators, modules, controllers, providers etc
Some experience programming in javascript and typescript

How nestjs serialization works

serialization in nestjs is enabled using the ClassSerializerInterceptor. It intercepts the responses and serialises them to the desired format before returning them to the client. The ClassSerializerInterceptor interceptor can be set at the application level to enable serialization by controllers -or at the service level for serialization by services.

An entity class (serializer class) is created to define the shape of the data one would like to return to the client. The class transformer package is then used to provide a set of rules to transform the data before it is returned to the client. The serialized data becomes an instance of the entity class and the returned value of a method handler as shown in the example below.

Example.

This example is a citizen registration app where citizens signup to become users by providing their information. It demonstrates serializing data by excluding some information provided such as password and includes the citizenStatus -a computation derived from the provided date of birth.

Set up:

To get started, in your terminal, install nestjs CLI if you do not have it installed already

npm install -g @nestjs/cli

Use the nest CLI to generate a new project boilerplate and select the package manager you'll be using

nest new citizen_app

next, install some packages that will be required for this example

npm i --save class-validator class-transformer

In the root of your project, create a userData.json file to serve as data storage for this project

touch userData.json

Rest Endpoints:

With the set-up complete, regenerate a resource for the user and select generate REST endpoints

nest g resource user

A new user folder should be added to your src folder containing, service, controllers, dto, entities etc

In the create-user.dto.ts, define the properties that will be used to create the user. Add some validations to the properties using the class-validator package as shown below:

import { IsNotEmpty, IsString, IsDateString, IsEmail } from 'class-validator';

export class CreateUserDto {
  @IsNotEmpty() //should not be empty
  @IsString() //should be a string
  firstname: string;

  @IsNotEmpty()
  @IsString()
  lastname: string;

  @IsDateString({ strict: true }) //should be in format 2022-07-15.
  dateOfBirth: Date;

  @IsEmail() //should be a valid email
  email: string;

  @IsNotEmpty()
  @IsString()
  password: string;

You have to enable global validations in your src/main.ts bootstrap() function for the validations to be applied. Add this code just before your app.listen()

app.useGlobalPipes(new ValidationPipe())

In the user folder, create a new folder called interfaces. create an interface for the created user

export interface IUser {
  id: number;
  firstname: string;
  lastname: string;
  dateOfBirth: Date;
  email: string;
  password: string;
}

In the src/user.service.ts, the UserService class will need to have access to the data storage file to read and write to it. To achieve that, The UserService class has to instantiate by reading the userData.json file as follows:

import * as fs from 'fs';
import * as path from 'path';
import {Injectable} from '@nestjs/commom';
import { CreateUserDto } from './dto/create-user.dto';
import { IUser } from './interfaces';

@Injectable()
export class UserService {
  private userData: IUser[];

  constructor() {
    fs.promises
      .readFile(path.join(__dirname, '../../userData.json'), 'utf-8')
      .then((data) => {
        this.userData = JSON.parse(data);
      })
      .catch((err) => {
        console.error(err);
      });
  }

  private saveUser() {
    fs.promises.writeFile(
      path.join(__dirname, '../../userData.json'),
      JSON.stringify(this.userData),
    );
  }
//the rest of the generated methods go here...
}

Still in src/user/user.service.ts, add some actual implementation to the rest of the method handlers.

//continuation of userService class...
//create new user
  create(dto: CreateUserDto) {
    const existingUser = this.userData.find((user) => user.email === dto.email);
    if (existingUser) throw new ConflictException('user already exists');

    const newUser: IUser = {
      id: this.userData.length + 1,
      firstname: dto.firstname,
      lastname: dto.lastname,
      dateOfBirth: dto.dateOfBirth,
      email: dto.email,
      password: dto.password,
    };
    this.userData.push(newUser); //add new user to userData
    this.saveUser();

    return newUser;
  }

// find all users
  findAll() {
    return this.userData;
  }

// find a user with id
  findOne(id: number) {
    const user = this.userData.find((user) => user.id === id);
    if (!user) throw new NotFoundException('user not found');
    return user;
  }

// delete user
  remove(id: number) {
    const userIndex = this.userData.findIndex((user) => user.id === id);
    if (userIndex === -1) throw new NotFoundException('user not found');
    this.userData.splice(userIndex, 1);
    this.saveUser();
  }

Note that in a real application, you'll have to secure your passwords before saving them by hashing them using a third-party package such as bcrypt

With the UserService fully implemented, start the server by running npm run start:dev in your terminal and send some requests. You can use Postman, Insomnia or any other tool of your choice.

The response from making a POST request to the user endpoint should look like this in Postman.

Notice that among other properties returned in the response object, we have password -which would compromise the security of your users' data. The plan is to have the password excluded from the response object. For this example, the dateOfBirth property will also be excluded and instead, a citizenStatus property -indicating whether the user is a child, adult or senior depending on their date of birth will be included.

Serializing the endpoints

The plan is to use the ClassSerializerInterceptor at the application level so that all the route handlers in the controller that are provided with an instance of the entity class (serializer class) can perform serialization.

In your src/main.ts, import the ClassSerializerInterceptor from @nestjs/common and add the interceptor globally. Your final bootstrap() function should look like this:

import { ClassSerializerInterceptor } from '@nestjs/common';
import { ValidationPipe } from '@nestjs/common';
import { NestFactory, Reflector } from '@nestjs/core';
import { AppModule } from './app.module';

async function bootstrap() {
  const app = await NestFactory.create(AppModule);
  app.useGlobalPipes(new ValidationPipe()); //set validations to the application level
  // 👇 apply transform to all responses
  app.useGlobalInterceptors(new ClassSerializerInterceptor(app.get(Reflector)));
  await app.listen(3000);
}
bootstrap();

Next, create an entity class and supply a set of rules for transforming the data. Add this to your src/user/entities/user.entity.ts

import { Exclude, Expose } from 'class-transformer';
import { IUser } from '../interfaces';

interface IUserEntity extends IUser {
  get citizenStatus(): CitizenStatus;
}

type CitizenStatus = 'Child' | 'Adult' | 'Senior';

export class UserEntity implements IUserEntity {
  id: number;
  firstname: string;
  lastname: string;
  email: string;

  @Exclude()
  dateOfBirth: Date;

  @Exclude()
  password: string;

  @Expose()
  get citizenStatus(): CitizenStatus {
    const birthDate = new Date(this.dateOfBirth).getFullYear();
    const currentYear = new Date().getFullYear();
    const age = currentYear - birthDate;
    if (age < 18) {
      return 'Child';
    } else if (age > 18 && age < 60) {
      return 'Adult';
    } else {
      return 'Senior';
    }
  }

In the provided snippet, the @Exclude() decorator from the class-transformer package is used to indicate that the password and dateOfBirth properties should be excluded from the response objects. On the other hand, the citizenStatus property is indicated to be included using the @Expose() decorator.

Since citizenStatus is not an actual property of the entity class -but rather a computed value based on the dateOfBirth property, the get keyword is used to define a getter method for citizenStatus instead of a regular property.

With the entity class created, the next step is to use it in the controller. Refactor your src/user/user.controller.tsto match this:

import { Controller, Get, Post, Body, Param, Delete } from '@nestjs/common';
import { UserService } from './user.service';
import { CreateUserDto } from './dto/create-user.dto';
import { UserEntity } from './entities/user.entity';

@Controller('user')
export class UserController {
  constructor(private readonly userService: UserService) {}

  @Post()
  create(@Body() createUserDto: CreateUserDto): UserEntity {
    return new UserEntity(this.userService.create(createUserDto));
  }

  @Get()
  findAll(): UserEntity[] {
    const users = this.userService.findAll();
    return users.map((user) => new UserEntity(user));
  }

  @Get(':id')
  findOne(@Param('id') id: string): UserEntity {
    return new UserEntity(this.userService.findOne(+id));
  }

  @Delete(':id')
  remove(@Param('id') id: string) {
    return this.userService.remove(+id);
  }
}

The controller's method handlers that will need to be serialized are refactored to return an instance of the entity class with the return values of the delegated service handlers. A return type of UserEntity is also added.

Restart your application and send some requests to confirm the serialiser is working as expected. The GET request to fetch all users provides the following response:

From the response, you can tell the data has been properly serialized to the desired format.

Conclusion

If you've made it to the end of this article, you've learnt how serialization works in nestjs and learnt hands-on to serialize data in nestjs. From the example, you can tell that among other things, you can return only a subset of an object and also make use of getters and setters.

There are so many other transformations you can apply to your response objects and this tutorial only scratched the surface with those. To dive deep into transformations, feel free to read more on the class-transformer Github. Also, check out the nestjs docs on this same topic.

The complete source code for this example can be found on Github here

Logging -and how to configure it in Python

Alwoch Sophia — Mon, 23 Jan 2023 10:03:42 +0000

As a developer, it is essential to stay on top of the changes in your application during development and production. Logging enables you to track events, features, errors and when they occur.

This tutorial will take you through setting up basic logging to advanced concepts and will cover:

what logging is
why logging is important
The python logging module
logging levels
creating loggers
storing logs
formatting logs
customising logs using the Colorlog package

Prerequisites

To follow along seamlessly, you may need to have the following:

a code editor such as VScode or an online IDE such as replit
basic python programming experience
python and pip installed if you're using a code editor

What is logging?

Logging is the process of recording information about an application's execution. This information may include and is not limited to:

events such as user actions, errors
performance metrics such as response times and memory usage
security events such as login attempts etc.

Why logging is important

Logging contributes significantly towards application health and here's why it should be configured:

Error tracking. Logging provides detailed information about errors, where and when they occur in your application making it easy for you to track and fix them easily.
Debugging. You can identify and fix bugs faster and more efficiently when you log information about your application's execution.
Auditing and monitoring. Logs provide information about how your application is being used and by whom.
Security. Logging information on user actions, system events and other security-related information enables you to notice security breaches and respond to them in a timely fashion.
Compliance. Regulatory and compliance requirements may require companies to provide a record of activity and changes to their systems which can be referenced using the application's logs

The Logging module.

The Logging module in Python provides classes and methods required to set up logging functionality for your applications. It is included as part of the Python standard library and can be used without installing it. To use the logging module, import it as follows:

#import the logging module
import logging

The above code avails the different methods and attributes in the logging module for you to use. You can explore these attributes and methods by calling the dir() method on the logging module and printing it to the console as shown below:

#print the logging module diretory
print(dir(logging))

The above code outputs a list containing log levels, methods and classes to the console as shown in the following image:

You can use the following criteria to make sense of the returned list:

Logging levels are indicated by uppercase letters. These are generally considered constants for example DEBUG, ERROR, CRITICAL etc
classes are written in pascal's case for example RootLogger, LoggerAdapter etc
methods are written in camel case for example getLogger, getLevelName etc
private attributes are indicated by preceding underscores for example _showwarning, _srcfile etc
special methods are indicated by preceding and proceeding double underscores.

While there's so much to work with in the logging module, this tutorial will use the logging levels, some of the classes and methods to show you how to configure logging for your application.

logging levels

Python uses logging levels to indicate the severity of a log message. This allows you to filter log messages based on their importance for example isolating critical logs to a different destination. The following table shows the log levels defined in the logging module, their numeric values and when you should use them.

In addition to the above constants, the logging module provides methods that can be used to indicate the severity of a log. These are lowercase versions of the above levels and are used by calling the method name on the logging object with a message as a parameter as shown in the snippet below:

logging.critical('This is a critical message')
logging.error('This is an error message')
logging.warning('This is a warning message')
logging.info('This is an info message')
logging.debug('This is a debugging message')

Running the above code produces the following result in the console

From the result in the console, you can observe that:

The INFO and DEBUG logs do not appear in the console. This is because the logging level is set to 30 -the numeric value for WARNING by default. The logging module only outputs logs for logging levels greater than or equal to the set level.
The logs are populated in the format level-logger name-message. The logger name is set to root by default -implying the root logger for cases where a logger name is not provided.
The output logs don't provide information on when the message was logged
The logs are output to the console because a log destination is not set.

The logs shown above don't provide much context into what is happening. The default behaviour can be improved by properly configuring the loggerfo

Configuring logging

The key steps involved in configuring standard logging include:

creating a logger
storing the logs
formatting the logs

creating a logger

Creating a logger provides a basis for which other configurations for the logger are built. The getLogger() method calls an existing logger by name or creates one if non-existent. Just below your import statement, copy and paste the following code snippet:

#create a logger
logger=logging.getLogger('basic_logger')
print(logger) #print the logger

The above code creates a logger and stores it in the logger variable. It also confirms the logger is created by printing the logger to the console. This should be your console result:

A basic_logger has been created as confirmed by the console result. However, the result also shows that the default log level is set to WARNING -implying only log levels greater than or equal to WARNING will be printed. For illustration purposes, we'll print all the log levels. To change the current set level to DEBUG, call the setLevel() function on the logger as seen in this snippet:

#create a logger
logger=logging.getLogger('basic_logger')
#set the current level to DEBUG
logger.setLevel(logging.DEBUG)

print(logger) #print the logger
print(logger.level) #print the level

Running the above code should confirm that the log level is set to DEBUG and this should be the result:

Now that you've successfully created the logger, you can now add additional configurations to the created logger variable

storing logs

The logging module provides the Handler class for determining where logs are stored. Several destinations may be used to store logs depending on the application and developer's needs. Some of these destinations include and are not limited to:

http: logs may be moved via POST request to a database
email: some developers may choose to have critical logs sent to an email
file: a file can be created on the server to which all the logs are populated.

A handler is created by initialising one or more destination handler classes and supplying it with a log level. A handler will only output logs for the specified log level or maintain the default level.

This example illustrates how to store logs in a file using the Filehandler class. Copy this snippet and paste it below the logger you created.

#create a file handler
file_handler = logging.FileHandler(filename='D:\\python\\pythonLogs.log',
                                   mode='w',
                                   encoding='utf-8')
#set the log level of the file handler
file_handler.setLevel(logging.DEBUG)
# add file handler to logger
logger.addHandler(file_handler)

Breaking down the code snippet:

A file handler is created by initializing the FileHandler class with a relative path to the preferred destination for the logs and assigned to the file_handler variable. In this example, the file handler writes the logs in write mode. if you'd like to preserve the history of your logs and not overwrite them, you can safely ignore the mode and maintain the default append mode.
The log level of the handler is set to DEBUG -meaning this handler will store all the log levels in this same file. if you'd like to store different log levels separately, you can create different handlers with different destinations for each of the log levels.
The handler is added to the logger

With the handler added to the logger, add the following test logs to your editor and run:

#print some example logging messages
logger.critical('This is a critical message')
logger.error('This is an error message')
logger.warning('This is a warning message')
logger.info('This is an info message')
logger.debug('This is a debugging message')

The result should be a pythonLogs.log file created at the specified destination with the logging messages as seen in the screenshot below:

Formatting logs

Up until this point, the printed logs only show the messages, however, information on the Log level and when the log was created would be useful.

The logging module provides the Formatter class for specifying the appearance of logs. The Formatter class is initialized with a format string created using the accepted log Record Attributes and passed to the handler to print the logs in the specified format, in the destination specified by the handler as shown below:

# create a formatter
formatter = logging.Formatter(
  '%(levelname)s- %(name)s-%(asctime)s - %(message)s')

# add formatter to file_handler
file_handler.setFormatter(formatter)

# add file handler to logger
logger.addHandler(file_handler)

In the above code, the formatter is created using the log level names, the log name, the time and the message. The formatter is added to the handler and the handler is then added to the logger.

The following code snippet uses a Dog class to demonstrate the functionality of the logging system we've created. The class is initialized using a dog name and logs the number of times a dog has been barked. If the dog is made to mark more than 3 times, an exception is raised. Add it to your editor and run

class Dog:
    def __init__ (self, dogname):
        self.dogname = dogname
        self.count = 0

    def bark(self):
        try:
            self.count += 1
            logger.info(f"{self.dogname} barked {self.count} times")
            if self.count > 3:
                raise Exception(f"{self.dogname} cannot bark more than 3 times.")
            else:
                print("Woof!")
        except Exception as e:
            logger.error(e)
            raise

logger.info('initialising the Dog class')
mendy=Dog('Mendy')
mendy.bark()
mendy.bark()
mendy.bark()
mendy.bark()

You should have the following results in your console and pythonLogs.log file respectively:

in the pythonLogs.log file, you can observe a change in the format of the logs and they now appear in the Level-log name-time-message format.

still, on Formatting, you may have noticed that python logs appear in a uniform colour and it may be difficult for a developer to distinguish between the logs, especially during development. A third-party package called colorlog improves this behaviour.

colorlog package

The colorlog package improves the built-in python logging module by enabling colour configurations for different log levels. It uses handers and formatters that work in a similar way to those in the logging module. The colorlog package is a third-party library and has to be installed to use it as follows:

pip install colorlog

after you've installed colorlog, go ahead and import it in your file as shown below:

import colorlog

with the colorlog methods now being available to use, copy and paste the colorlog configurations below into your editor:

# Create a colored formatter
formatter = colorlog.ColoredFormatter(
    '%(log_color)s%(levelname)s:%(name)s:%(message)s',
    log_colors={
        'DEBUG': 'cyan',
        'INFO': 'green',
        'WARNING': 'yellow',
        'ERROR': 'red',
        'CRITICAL': 'red',
    }
)

# Create formatted logger
log = colorlog.getLogger('formatted_logger')
log.setLevel("DEBUG")

# Add a handler to the formatted_logger
handler = colorlog.StreamHandler()
#add formatter to handler
handler.setFormatter(formatter)
#add handler to logger
log.addHandler(handler)
# Log some messages
log.debug("This is a debug message")
log.info("This is an info message")
log.warning("This is a warning message")
log.error("This is an error message")
log.critical("This is a critical message")

The code snippet above creates a formatter and adds log colours to the format string as alog_colors attribute. The log_colors dictionary contains key value pairs of the log levels and the colors they should appear in. A StreamHandler is initialized to create the logs in the console. The formatter is added to the handler -and the handler is then added to the logger. when you run this code, you should have coloured logs appear in your console as in the screenshot below:

Conclusion

You've learnt how to configure logging using the logging module and how to add colour formatting to your logs using the colorlog package. The article illustrates only one of the ways logging can be configured using the logging module. You can challenge yourself to use a YAML file where you want to use multiple handlers or the basicConfig() method where you need just a single handler. Also, you can explore using the various handlers to send logs to destinations that were not illustrated in this article. You can check out the python docs for more on logging and find the source code used for this article on replit here.