Puppeteer and Selenium are both popular, powerful and widely used solutions for automating web applications.
But which one should YOU use?
The answer depends on your specific needs.
Choosing the wrong option could delay or worse, sabotage your automated testing project.
I'll share my own opinion at the end of this article.
Here are some PROS and CONS for both of them.
Puppeteer
PROS
• Faster than Selenium
• Easy to install
• Written and maintained by the Google Chrome devs
CONS
• Works only with Chrome
• The only supported language is Node.js
Selenium
PROS
• Works with most browsers
• Multi-language support
• Huge community of users
CONS
• Difficult to run on all browsers
• A bit slower than Puppeteer
To sum it up, Puppeteer is faster than Selenium, but it works only with Chrome, while Selenium works with Chrome, Firefox, Safari, Internet Explorer and Edge.
Now, for my opinion: both solutions are outdated and built with the narrow mindset of developers who refuse to see beyond their code.
It may sound lazy, but I believe that a solution for automated testing should be like a washing machine: it should give me enough flexibility to choose different inputs and settings, but it shouldn't make me pump the water with one hand and spin the drum with the other.
What both solutions are missing:
• Video Recording functionality
• Turnkey Machine Learning
• Integrated Email Testing option
• Screenshot Comparison feature
• Automatic Smart Waiting
• Structured results
Other issues with Selenium and Puppeteer:
• Difficult to upload files in a test
• Difficult to deal with iframes
• Difficult to integrate with your CI/CD system
• Difficult to configure Element Load Timeout
If you're looking for a better alternative, try Endtest.
It has Puppeteer's speed, it supports Selenium's browsers AND it has all the features that they're missing.
Top comments (21)
@klaus ,
your post sounds quite biased.
1) Every now and then (for the last 20 years) a new tool pops up that promises easy to use test specification without any coding knowledge needed. None of those tools have lived up to their promises.
2) You should compare your tool with Serenity BDD, CodeceptJS or any other high level test suite. Selenium and Puppeteer are merely the drivers that speak with the browser
3) Given that more and more teams learn to work agile, testers will be able to code and coders will be able to test. That renders a dedicated tool for people that can not code useless
4) I can code. I have experimented with a few point and klick test tools. Your approach might be of some use for tiny test suites which have no relevance in the real world. Once a test suite becomes a bit bigger there is no alternative to code tests.
Thank you for taking the time to write that comment.
CodeceptJS and Serenity BDD are just some libraries.
Selenium is also just a library.
The open source and zero license cost nature of such libraries is attractive when compared to the license or subscription costs of commercial automation solutions.
But there is more to it than just that when you look at the cost of implementing an automation solution.
For example, Selenium is open source and free, however building and using an automation solution on this ‘free’ software comes with some real costs:
1) The open source tools/libraries provide a starting point but you still have to write code for test creation, management, execution, handling errors and reporting. Note that this is over and above the test logic that you want to automate.
2) If you are looking to test integrations with back-end systems and external systems via API, you have to build in support for that into your framework.
3) If you are looking to implement a continuous testing solution, you must add to the above effort, the effort of building integration with DevOps or CI/CD tools.
4) Also to be considered are the costs of building and managing an infrastructure to run these tests. These costs are likely to go up as you scale your infrastructure in order to support increased test volumes or to increase the pace of testing.
To be honest, "EndTest" got nothing to do with the comparison between "Puppeteer vs Selenium". You are missing a big part of Puppeteer and Selenium.
EndTest is focused on testing in mind. You should compare testing tools when mentioning EndTest, not browser automation tools. Your tool is not a browser automation tool, directly.
Nope, EndLess is completely out of context here.
Hey, your link above isn't working for me - SSL error. Just FYI
I don't know which link you're talking about, all of them work.
oh, okay. I've retried it and it worked. I ended on the www version (which didn't work). Anyways. Interesting idea
Interesting.
The www version redirects me to the non-www version.
What browser are you using?
Firefox 68. I've just tested it and it doesn't redirect www to non-www for me.
Thank you Klaus, Great article! I liked that you briefly explained the pros & cons of Selenium and Puppeteer. Also, I liked the reference to what was missing from both solutions. I agree that Puppeteer is faster than Selenium only with Chrome, while Selenium works with Chrome, Firefox, Safari, Internet Explorer and Edge (Don't forget also Mozilla). See this table that explains very nicely the differences you have indicated(See pic) from blog.testproject.io/2020/02/20/sel...
There is not that much point with no-code autotests since even very extensive autotests only require high level script coding. The challenges are not primarily programming techniques / advanced coding.
github.com/GoogleChrome/puppeteer/...
It says:
BEWARE: Experimental. Just for preview. Installation and usage will change.
We can talk about it when it's production ready.
Still, it got featured on Chrome's YouTube changelog, so that may indicate some measure of stability
It's an interesting implementation of a Keyword/Action driven automated testing framework. I've built similar using Java, Selenium, SauceLab and a few add-on libraries for screenshot comparisons. I could see the pull your platform would have on some QA organizations or Directors of Engineering.
Since you've put your product out here I'm assuming you're cool if I ask questions about your platform.
1) I've reviewed your documentation and what I didn't see was how your test framework handles executing the same tests against different environments or URLs. Current software dev shops have multiple environments that they want to run their automated test against (Dev, QA, Stage, ect). How is this handled in your platform using variables?
2) It would be nice to see a demo of how you can have 1 test or suite execute against against multiple browsers without having to duplicate tests for each browser.
3) Do you have any comprehensive examples of your structured results reporting? Your documentation is lacking in this area and as we all know, upper management folks like pretty pictures. ;-)
4) Does your platform offer a way to call external APIs or programs? When I build my automation for a GUI, I like to pre-populate my tests with data before the UI executions and validations without having to drive the UI all the time. Is this possible?
5) When you integrate your platform with Jenkins, Github or others, are the test results pushed back to the calling platform?
6) With your platform retaining all of test actions, video recordings, screenshots and basically the keys to a software teams kingdom with your constant analyzing the sites/apps, what is your security model to protect your customers data?
Thanks for letting me prob into your platform.
Cheers!
Ken
Hi Ken,
I really appreciate the fact that you tried Endtest.
My opinion is that you could have obtained some of those answers simply by digging more into the platform.
But here are the answers:
You can easily run tests on different environments (Dev, QA, Stage, etc) with Endtest.
Those environments are obviously going to have different URLs, different usernames, different passwords, etc.
You can use the "Set Variable" action to create or generate different sets of variables and you can use IF Statements to see which environment you're on and to use the corresponding usernames and passwords.
You can even define variables directly in the API call for starting a test execution, which makes it even easier.
There are multiple ways in which you can handle different environments, it depends on what works best for you.
You can run the same test suite on any operating system, browser or mobile device.
All you have to do is select the desired configuration in the "Run Test Suite" modal or in the API call.
You don't have to duplicate or modify a test suite in order to run it on different configurations.
You can export the test results directly from the Results page. In that XLSX report, you have all the numbers, all the details, everything that can be extracted from a test execution.
Since it's in XLSX format, you can easily create your own reports in Microsoft Excel by simply selecting the relevant columns.
Yes, you can use the "Execute JavaScript" step to call any API you want or you can use the "Connect to Database" and "Execute SQL Query" actions to write that data directly in your database.
Absolutely, the integration with services like Jenkins and Github is done through webhooks, you just add the API calls for starting a test execution and fetching the results in the Webhooks sections from those services.
Additionally, Endtest provides it's own Webhook functionality, this allows you to automatically send the results to a URL after each test execution.
All the traffic on the Endtest platform is encrypted.
The test data from the execution machines is deleted right after the test is finished.
The Machine Learning functionality from Endtest does not constantly analyse the sites and the apps, it only analyses them while the tests are being executed.
There are multiple Security Standards that are being followed, including the famous ISO/IEC 27001.
Klaus,
Thanks for the quick response. It was very helpful.
Hi!
Just wanted to say that you have a typo in selenium cons:
should be A bit slower than Puppeteer ?
Thank you for pointing that out. Fixed it.
Your "article" should start with a conflict-of-interest disclosure...
As Puppeteer now supports Firefox in addition to the Chrome browser, you guys might want to take a look at this excellent learning resource on Puppeteer written by Dario Kondratiuk - amazon.com/Testing-Puppeteer-end-e...
Ads are getting smarter huh?