What's up DEV Network?
In this post I want to share my automation experience from my current internship.
Youtube Channel with video tutorials - Reverse Python Youtube
But before we dive in please visit Reverse Python if you want to find more articles like this.
Actually, I wanted to explain it to you on a real project, but unfortunately, I didn't find any platform that fits all my requirements.
However, if you are Python developer you will need these tips to build your automation program fast. So, let's start!
It is important to configure web driver correctly to be able to run automation. If you want to use Chrome as a web driver then you should install chromedriver. However, if you want to choose Firefox then you should install geckodriver.
Creating a class
With creating a class you can easily handle URLs and call functions so you don't have to create web drivers in each function. Take a look following code:
from selenium import webdriver class Bot: def __init__(self): self.bot = webdriver.Firefox(executable_path='/path/to/geckodriver') automate = Bot()
So, when you need driver in function just write:
def search(self): bot = self.bot bot.get('www.google.com')
For handle login you should add fields into __init__ function. Take a look following code:
import time from selenium import webdriver from selenium.webdriver.common.keys import Keys class Bot: def __init__(self,username,password): self.username = username self.password = password self.bot = webdriver.Firefox(executable_path='/path/to/geckodriver') def login(self): bot = self.bot bot.get('https://website.com/') time.sleep(3) email = bot.find_element_by_id('userNameBox') password = bot.find_element_by_id('passWordBox') email.clear() password.clear() email.send_keys(self.username) password.send_keys(self.password) bot.find_element_by_id('LoginBtn').click() time.sleep(3) automate = Bot('your_username', 'your_password') automate.login()
Very simple approach to handle logins.
It is important to use time.sleep() if you are looping clicks. Because automation needs time to perform another click. If you don't use time.sleep() then ElementClickInterceptedException error will appear.
for elem in elements: elem.click() time.sleep(3)
Write data into CSV
There is a lot of solutions and examples on the internet about how to write data to CSV. Let's assume that you have multiple arrays and you want to write them into CSV under the right field names (column name).
Let me be more clear..
My task was to crawl some phrases and translate them to all languages then write all these data into CSV. So, I used GT API and created an array for each language.
I had a lot of arrays so I need somehow write each translation under right field names (column name).
The following code demonstrating the best solution to handle these kinds of problems:
# zip arrays languages_translations = zip(ar,hy,ms,bg,zh_cn_SAR,zh_cn, zh_tw_singapore,zh_tw,hr,cs,da,nl,en_australia,en_uk,en_usa,et,fi) # Write to CSV with open('translations.csv', mode='w', newline='', encoding="utf-8") as csv_file: fieldnames = ['Arabic', 'Armenian (Armenia)', 'Bahasa Malaysia (Malaysia)', 'Bulgarian (Bulgaria)', 'Chinese (Hong Kong SAR)', 'Chinese (Simplified)','Chinese (Singapore)','Chinese (Traditional)','Croatian (Croatia)', 'Czech (Czech Republic)', 'Danish (Denmark)','Dutch (The Netherlands)','English (Australia)','English (UK)', 'English (US)','Estonian (Estonia)','Finnish (Finland)'] writer = csv.DictWriter(csv_file, fieldnames=fieldnames) writer.writeheader() for lang in languages_translations: writer.writerow(dict(zip(fieldnames, lang)))
As you see, you should zip all arrays, loop it and write rows dictionary of zipped field names and loop items.
By this way you will write all data correctly into CSV file.
Handle Popups or Iframes
Another challenging task is to handle popups or iframes while automation. If you need to interact with popup or iframe elements, you should tell selenium to switch the window from main to popup. Once you finished with popup you should switch back to the main window.
# find iframe or popup element iframe = bot.find_element_by_tag_name('iframe') # switch to iframe or popup window bot.switch_to.frame(iframe) # interact with elements textarea = bot.find_element_by_tag_name('body') textarea.send_keys('Some Keys Here') # switch back to main window bot.switch_to.default_content()
Great! I am providing you the solutions which is really complicated on internet.
If you have to handle alert popups you also can use this method.
Send CSV data to web elements
Sometimes you need to send csv data into web elements such as multiple textboxes.
Assume that you have a multiple panels with multiple textboxes in it and you have to send CSV rows into these texboxes
So each CSV row belongs different panels. Sounds crazy right? This is how you will train your brain :) Somehow you have to iterate each CSV row for each panel's textboxes.
Before we go on please visit Reverse Python if you want to find more articles like this.
Alrgiht! Let's see the code and then I am going to explain:
def send_keys_textboxes(self,url): bot = self.bot # go to url bot.get(url) # reader object which will iterate over lines in the given csvfile with open('translations.csv', 'rt', encoding='utf8') as csvfile: langs = csv.reader(csvfile, delimiter=',', quotechar='"') # create list of langs langs = list(langs) elements = bot.find_elements_by_xpath("//a[@data-tag='globalize']") # Using index to handle multiple panels index = 0 try: for elem in elements: class_of_element = elem.get_attribute("class") if class_of_element == 'cs-trans-icon': # Panel opens with click elem.click() time.sleep(3) textBoxes = bot.find_elements_by_tag_name('textarea') # Loop only specific index of list phrases = langs[index] # Itearating TextBoxes for i in range(len(phrases)): textBoxes[i].send_keys(phrases[i].title()) time.sleep(3) # Increasing index for next panel index = index+1 try: bot.find_element_by_class_name('CsImageButton').click() except NoSuchElementException: bot.find_element_by_class_name('cso-btn').click() time.sleep(3) except ElementClickInterceptedException: pass
Index plays main role in this solution. We said that we want to handle multiple panels, so index prevents to loop all rows in each panel. Assume that you have 3 rows and without using index, program will continue to loop until 3rd row finished. As a result you will have 3 values for each textbox. However, we need to send only first row values into first panel's textboxes, second row values into second panel's textboxes and so on..
Once index is defined, we are iterating textboxes. By this way, first value in row is going to send first textbox, second value in row is going to send second textbox and so on..
These examples are from the real-world project so I recommend to bookmark this post.
So there are 2 approaches for handling dropdwons.
First solution is to use selenium's Select method to select options from dropdown.
from selenium.webdriver.support.ui import Select bot = self.bot bot.get("https://example.com") select = Select(bot.find_element_by_xpath("//select")) # select bu index of option select.select_by_index(2) # select by text of option select.select_by_visible_text('Visible Text') # select by value of option select.select_by_value('value')
and second solution is just using xpath to click dropdown items:
Navigate parent element with xpath
Sometimes Html structures can be messed up and you can't select the element you want. In these cases, it is good to navigate the parent element or go back using xpath:
# elem is a web element object # selenium will jump 2 step back to look for parent element parent_element = elem.find_element_by_xpath('..').find_element_by_xpath('..')
That is all for now! I shared my python automation experience with you and I hope it helps you and saves your time from searching on the internet.
Please visit Reverse Python for more articles (you won't disappointed) and let me know in comments what my next post should be about.
See you very soon DEVs! Stay Connected!