About 2+ years ago - the initial prototype of the language took about 6 months to make from scratch, a very large part of it done by my co-founder @shiling
.
To be fair, webdriver protocol in itself is already a miracle. Which is what we depend on.
We then made our own JS test library that gets injected into each browser, to scan the website, which then communicates back to our server to when performing specific steps (like button presses)
It, however had one very big catch, it only worked only on chrome, and on about 50% of the website frameworks out there. Unless you were one of those who followed the HTML guidelines perfectly with aria labels and all.
(It also came along with the very terrible v1 of our backend infrastructure. Where we were burning cloud credits by giving every users dedicated servers - but that's another story... haha)
Fast forward 2 year. And as more time was spent into refining the engine, we managed to cover all major UI framework and design patterns within our system. So as much as there are a 100+ ways to style a button, we would detect and test accordingly.
At the same time, coverage been extended to Firefox, Safari, Edge, and IE 11.
And we have long since ran out of cloud credits, and restructure our entire system as a shared infrastructure (except for those deploying on-premise)
Problems faced? Oh boy, many. The earliest was getting our test engine to deal well with human language and ambiguity, especially how it decides which login button to click on if there are 5 of it on screen.
Alot of work is put into making our system be able to understand user testing patterns, and acting accordingly. Without an AI. More is covered by my co-founder here : dev.to/uilicious/taking-the-horror...
From then on - lots of infrastructure issues..
Especially with our underlying technology selenium/webdriver. It's hilarious and sad, how in some selenium testing lessons, they teach users to run one test 4 times, and if any 1 pass. Consider it as a pass. And that's due to the huge failure rate of the system by default. Which is honestly not acceptable. So lots of effort is then put into configuring selenium to be stable enough for reliable use.
It also doesn't help when all the browser vendors keep breaking their webdriver specs. Amusingly, IE11 now is increasingly getting more love from our team, for never changing haha.
Oh, and to find out after you thought you got all 100 ways of testing a login page working... to find a website that does something completely different 😂
O_O This was a great story, it should've been shared on actual post.
I certainly faced almost all problems and had to tackle in different ways you have mentioned while working on various web automation projects. Even till now I haven't finished solving all those public problems stated on stackoverflow.
Haha, thanks, it certainly will be fleshed out into its own post one day. Especially once we dive into the details.
Its great to see others tackling this problem as well, as I seen many have given up on automation out of frustration sadly. We have yet to solve most of our problem like those you face as well after all, part of the reason why we consider "IE11, Safari, and edge" beta right now.
For further actions, you may consider blocking this person and/or reporting abuse
We're a place where coders share, stay up-to-date and grow their careers.
Cool!
Any story to share?
:D
About 2+ years ago - the initial prototype of the language took about 6 months to make from scratch, a very large part of it done by my co-founder @shiling .
To be fair, webdriver protocol in itself is already a miracle. Which is what we depend on.
We then made our own JS test library that gets injected into each browser, to scan the website, which then communicates back to our server to when performing specific steps (like button presses)
It, however had one very big catch, it only worked only on chrome, and on about 50% of the website frameworks out there. Unless you were one of those who followed the HTML guidelines perfectly with aria labels and all.
(It also came along with the very terrible v1 of our backend infrastructure. Where we were burning cloud credits by giving every users dedicated servers - but that's another story... haha)
Fast forward 2 year. And as more time was spent into refining the engine, we managed to cover all major UI framework and design patterns within our system. So as much as there are a 100+ ways to style a button, we would detect and test accordingly.
At the same time, coverage been extended to Firefox, Safari, Edge, and IE 11.
And we have long since ran out of cloud credits, and restructure our entire system as a shared infrastructure (except for those deploying on-premise)
Problems faced? Oh boy, many. The earliest was getting our test engine to deal well with human language and ambiguity, especially how it decides which login button to click on if there are 5 of it on screen.
Alot of work is put into making our system be able to understand user testing patterns, and acting accordingly. Without an AI. More is covered by my co-founder here : dev.to/uilicious/taking-the-horror...
From then on - lots of infrastructure issues..
Especially with our underlying technology selenium/webdriver. It's hilarious and sad, how in some selenium testing lessons, they teach users to run one test 4 times, and if any 1 pass. Consider it as a pass. And that's due to the huge failure rate of the system by default. Which is honestly not acceptable. So lots of effort is then put into configuring selenium to be stable enough for reliable use.
It also doesn't help when all the browser vendors keep breaking their webdriver specs. Amusingly, IE11 now is increasingly getting more love from our team, for never changing haha.
Oh, and to find out after you thought you got all 100 ways of testing a login page working... to find a website that does something completely different 😂
O_O This was a great story, it should've been shared on actual post.
I certainly faced almost all problems and had to tackle in different ways you have mentioned while working on various web automation projects. Even till now I haven't finished solving all those public problems stated on stackoverflow.
Haha, thanks, it certainly will be fleshed out into its own post one day. Especially once we dive into the details.
Its great to see others tackling this problem as well, as I seen many have given up on automation out of frustration sadly. We have yet to solve most of our problem like those you face as well after all, part of the reason why we consider "IE11, Safari, and edge" beta right now.