Using PyAutoGUI to Automate Your Python Scripts

Project Day

Hello! Today I completed phase 3 of 5 of my full-stack software engineering course here at Flatiron School. As with every phase, we were tasked with submitting and presenting a final project. This phase, we were tasked with building a CLI application using python and SQLAlchemy ORM. Excited to put our skills to work, my partner and I brainstormed project ideas. Ultimately, we settled on creating a application that mimics the functionality of a Command Line Interface.

Getting Started

With python's built-in os module, allowing the user to input commands to navigate and interact with the files within our SQLite database was simple enough.

For example, this is was our code to cd (change directory) based on user input:

import os 

#instantiating terminal while program is active 
terminal_active = True

while terminal_active == True: 
#check if user input is valid 
if x.split(" ")[0] == "cd":

#cd into user's specified directory 
    os.chdir(x.split(" ")[1])

Easy enough.

The Problem: Opening and Playing a Youtube Video

While implementing the "open" command (i.e. open file), I decided to add an Easter egg: if the inputed file does not exist, a youtube video (Rick Roll - Never Gonna Give You Up) is opened and played in the browser. By use of another python module webbrowser, opening a video in the browser was simple enough:

import webbrowser

webbrowser.open("https://www.youtube.com/watch?v=dQw4w9WgXcQ")
## This will open the file in the user's default browser

However, the tricky part was getting the video to then play after being opened. Sifting through dozens of outdated stackoverflow posts yielded no promising results. Many proposed solutions were outdated, and others proved too cumbersome to implement.

The Solution: Automate

At last, I stumbled upon PyAutoGUI and a solution was in sight. PyAutoGUI is an installable package which allows you to use Python scripts to directly control the mouse and keyboard. Thus, use of PyAutoGUI makes for seamless automation of application-to-application interactions.

PyAutoGUI Functions

For my particular problem, three of PyAutoGUI's functions came in handy:

1. size()
2. moveTo()
3. click()

The size function returns a named tuple with the resolution (width and height) of your screen:

width_height = pyautogui.size() 

>>> width_height

#return value
Size(width=1920, height=1080)

With the size function, I was able to determine the x,y coordinates center-screen. Next, it was only a matter of using moveTo() and click() to move the cursor to the center of the screen and click "play":

import webbrowser
import pyautogui
import time

webbrowser.open("https://www.youtube.com/watch?v=dQw4w9WgXcQ")

#wait five seconds before moving cursor
time.sleep(5)

# moves cursor to center of (my) screen
pyautogui.moveTo(700, 450, 2)      

# clicks cursor after .05 seconds
time.sleep(.05)
pyautogui.click()

Success!

Et voilà! With this code, my program successfully opened and played the youtube video. While my use of PyAutoGUI's automation was quite simple, it nonetheless serves as an example of the enormous potential automation has in the world of software engineering.

Thanks for Reading!