Ever wanted to know as soon as
- a product comes into stock?
- tickets are available for any event?
I think most of us have had those questions. After reading this, you'll know how to implement answers for the above questions/similar questions.
This implementation had paved ways to find answers to the above questions.
The project mentioned above was built in last summer(2019 April) during the IPL season. What is IPL? check here.
The main objective of this project is to notify the user when someone hits a SIX in the IPL match. With few modifications, you can build a system to notify yourself when an event occurs in a website.
Diving into how I did it, the first and core concept I should discuss about is Web Scraping. Web Scraping allows you to get data from websites even if the website owner is not providing any API kind of thing. I used Python language in this project and will be talking about python packages throughout this post. Python has web scraping libraries like BeautifulSoup, Scrapy etc. I used BeautifulSoup here because it is simple and fits our need.
BeautifulSoup consists of powerful parsers that can parse response objects into traversable objects. It also has several inbuilt methods to filter out the data extracted from the website. We can achieve a lot of things using them.
So, you first need to make a request to the website you wanted to scrape using its URL. This can be done with packages like urllib, urllib3, requests etc. You'll get a HttpResponse object from the server of the website you requested. Now you have to pass that response object to BeautifulSoup class specifying which parser to use. It will give a parsed bs4 object accordingly.
Now, you have to check for the required data from the parsed object using builtin methods or regular expressions.
You'll want to notify after checking for the data. To notify the user via email, you need two packages: smtplib, email.
email is used to frame email object which we will be using in sendmail method of smtplib.
Using threads to send notifications can prevent the pausing of main activity.
Top comments (0)