Recently I had a requirement of implementing a web scraping tool to identify changes in a website. It was the virtual learning environment website of my university where I wanted to create a desktop notification whenever our examination results were uploaded to the website. I started by writing a Python script that checks the website and creates a notification on Windows if updates are detected. I used the Python Beautiful Soup library to parse the HTML content of the web page and the Windows 10 Toast Notifications library to generate Windows 10 toast notifications. These tools made the implementation pretty simple.
The next phase was to run the script every 10 minutes and my initial thought was to use
time.sleep() while the script was running in an infinite loop. That's when I started to realize that even though this approach solves the problem, using
time.sleep() does not mean that the code is executed every 10 minutes. What it does is put a 10 minute gap between executions. For a simple expendable application(like what I'm implementing) this is not an issue but for mission-critical applications or streaming applications using
time.sleep() for scheduling might have adverse effects. So I decided to use Windows Task Scheduler to run my Python script every 10 minutes.
Steps to schedule a Python script using the Windows Task Scheduler
Step 1 - Create a batch file to run the Python script
Open Notepad, add the following syntax and save it with the
start "" "path_to_python.exe" "path_to_python_script"
start "" "C:\Python38\python.exe" "C:\Users\Tharinda\Desktop\web_scraper\main.py"
start makes sure the cmd is closed after the execution. The first "" is the title that should be used with
start and in this case, the title is left blank.
After double-clicking on the .bat file the python script should run if the file is configured properly.
Step 2 - Create a task in Windows Task Scheduler
start menu and search for
- In the
- Give a name for the task
- Switch to the
Triggerstab and press
Newto create a new trigger.
- Configure the trigger to suit your needs. In my case, the trigger is scheduled to run daily repeating the task every 10 minutes.
- Switch to the
Actionstab and press
Newto create a new Action.
Program/Scriptadd the batch file that was created in step 1
Add Conditions and change the Settings depending on your requirement and press
OK to save the task. You can refer the Task Scheduler documentation to learn more about the Windows Task Scheduler.
That is pretty much it. Your task should run as intended.
I hope this helps. Cheers!
Top comments (1)
Thank you it helped me a lot, also make sure if your files use other files; cd to the folder containing them first and then run the script :)