Measuring Python Code Execution Time

Focus Ifeanyi — Fri, 10 Apr 2020 15:43:27 +0000

Timing how long it takes for a code snippet to run is a good way to measure the performance of your code. Though this doesn't really give the accurate measurement of the performance of your code snippet, but it gives you an idea of the increase or decrease in the performance of your code snippet.

In this short post, I'll be giving you tips on how to use the timeit python module in measuring your python code snippet.

timeit is a built-in Python module that provides a simple way to time small bits of python code. This Module can be used from the Command-Line Interface(CLI) or can be imported to your code and called.

Using timeit within your code

import timeit


def myFunction():

    for i in range(1000):
        print (i*i)

print(timeit.timeit(myFunction, number=1))

In the code, we are looping through a range of 100 and printing the multiple of each step. Then we printed the time it took the code to run once(number=1) using the timeit method of the timeit module. The function takes to arguments:

The name of the function to time
The number of times we want to run the run the function. When the number of time is set, it returns the time it took to run the function in the given number of times.

If we want to know how long it will take for the function to run 10 times, then we have to specify that in the number argument:

import timeit


def myFunction():

    for i in range(1000):
        print (i*i)

print(timeit.timeit(myFunction, number=10))

It's important to note that the output of running this code would be the number of times it took to run the myFunction function in seconds. If you would like to get home times it took to run once, you can set the value number argument to 1 or divide by the value of number set.

This is just the basic use case of the timeit module. There are other methods available on the timeit modules and can be found in the python module documentation

Hope this was helpful, feel free to contribute and let me know if I missed something.

Building a COVID-19 Data Alert System with Python and Selenium (Part 1)

Focus Ifeanyi — Wed, 08 Apr 2020 09:12:03 +0000

I got a bit bored during the COVID-19 lockdown here in Nigeria, so I decided to play around with data scraping using Python and Selenium.

The first question I needed to answer was what data I would like to scrape, and what I would be doing with the scraped data.

I decided to build a system that scrapes COVID-19 data from NCDC website, and sends me an SMS notification once there's an increase in cases within the country.

In this series, I'll be sharing how I built this simple system, hopefully, it would help someone build something larger and better.

Setting Up

We'll be using Selenium Python library, and a browser driver (either Chrome or Firefox driver).

Install Selenium

   pip install selenium

Download and Install Browser Driver

You can download either of the drivers from the links below:

Let's Go!

Now that we have installed our library and browser driver, it's time to get into the code.

Create your .py file and write:

from selenium import Chrome
from selenium.webdriver.common.by import By

# Make an instance of the chrome driver
driver = webdriver.Chrome()

# Initiate the browser and visit the link
driver.get('https://covid19.ncdc.gov.ng/')

In the code above we are importing Chrome driver from selenium and also importing 'By' private method from selenium web driver for selecting how we would like to locate elements. Also, we are initiating the browser using Chrome driver and using it to visit the page to be scraped.

If you prefer to use Firefox driver, your code would look like this:

from selenium import Firefox
from selenium.webdriver.common.by import By

# Make an instance of the firefox driver
driver = webdriver.Firefox()

# Initiate the browser and visit the link
driver.get('https://covid19.ncdc.gov.ng/')

The next step would be to locate the element where these data are, it's important to study and inspect how the DOM of the website to be scraped is rendered. For our NCDC Covid-19 the data we want is located within a table with id custom3.

table = driver.find_elements(By.XPATH, "//table[@id='custom3']/tbody/tr")
print(table[1].text)

Here we are finding elements by XPATH, there are so many other methods of finding elements and you can find the available methods on the Selenium Python Documentation. We find the table tag with id of custom3, then go into the tbody and finally the tr tag, this is possible using XPATH. We then print the second content to see what we have, before we proceed.

On our terminal we will change directory to our project root folder run:

python your_file_name.py

We will get this as a result:

We can see we have been able to get some data from the page, we will proceed to get the header of the table.

header_tag = table[0].find_elements_by_tag_name('th')

headers = list(map(lambda header: header.text, header_tag))

print (headers)

The code above finds the table header(th)tags within the first row and adds them to a header list. Running the script will print a list of the headers:

Finally, we will get the data for all the states:


results = []


for item in table:
    data = item.find_elements_by_tag_name('td')
    sd = list(map(lambda item: item.text, data))

    results.append(sd)

#Always remember to quit the browser driver when you are done
driver.quit()

results =  list(filter(None, results))
print(results)

Here we created an empty list called results, iterated through the table row element and searched for element by tag name 'td'. Within each table row, we have our data for each state. Then we finally filtered out empty rows and printed the results.

Voila!!! We have the COVID-19 data across different states in Nigeria.

Now that we have our data ready, in the next part of this series we will be saving the data to some kind of database for easy retrieval and tracking of changes.

Our entire code so far looks like this:

from selenium import webdriver
from selenium.webdriver.common.by import By
import itertools
import csv


driver = webdriver.Chrome()

# Start the browser and visit the link
driver.get('https://covid19.ncdc.gov.ng/')

table = driver.find_elements(By.XPATH, "//table[@id='custom3']/tbody/tr")

header_tag = table[0].find_elements_by_tag_name('th')

headers = list(map(lambda item: item.text, header_tag))

results = []


for item in table:
    data = item.find_elements_by_tag_name('td')
    sd = list(map(lambda item: item.text, data))

    results.append(sd)

#Always remember to quit the browser driver when you are done
driver.quit()

results =  list(filter(None, results))

print(results)

I hope I've been able to share my process well enough in this first part, feel free to contribute if you feel something is missing, or there are better ways to have done certain things. I look forward to your contribution(s). Catch ya in the next part.

DEV Community: Focus Ifeanyi

Measuring Python Code Execution Time

Using timeit within your code

Building a COVID-19 Data Alert System with Python and Selenium (Part 1)

Setting Up

Let's Go!