DEV Community: gcalvoCR

Allure and Jenkins

gcalvoCR — Tue, 20 Oct 2020 22:57:50 +0000

In order to use Allure inside Jenkins you basically need 3 things:

Under Jenkins > Manage Jenkins > Manage Plugins install the Allure plugin.

Then, under Jenkins > Manage Jenkins > Global Tool Configuration you also need to install the Allure Commandline. In order to do so, you can try to installing it automatically from the Maven Central Repo, as follows:

In each project, set the path to the allure-results directory.

That's it, we're ready to generate Allure reports.

The first time you run your job, Jenkins will try to download the installation zip file and install the Allure Commandline for you.

If for some reason it gives you a problem similar to the following:

[Checks API] No suitable checks publisher found.
java.io.IOException: Server returned HTTP response code: 401 for URL: https://repo.jenkins-ci.org/maven-repo1/io/qameta/allure/allure-commandline/2.13.6/allure-commandline-2.13.6.zip
    at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1900)
    at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1498)
    at sun.net.www.protocol.http.HttpURLConnection.getHeaderField(HttpURLConnection.java:3061)
    at java.net.URLConnection.getHeaderFieldLong(URLConnection.java:628)
    at java.net.URLConnection.getContentLengthLong(URLConnection.java:500)
    at java.net.URLConnection.getContentLength(URLConnection.java:484)
    at sun.net.www.protocol.https.HttpsURLConnectionImpl.getContentLength(HttpsURLConnectionImpl.java:412)
    at org.jvnet.robust_http_client.RetryableHttpStream.<init>(RetryableHttpStream.java:90)
Caused: java.io.IOException: Server returned HTTP response code: 401 for URL: https://repo.jenkins-ci.org/maven-repo1/io/qameta/allure/allure-commandline/2.13.6/allure-commandline-2.13.6.zip

The problem is that you're trying to connect to https://repo.jenkins-ci.org and you probably don't have permission.

What you could do is try install the Allure Commandline Jenkins > Manage Jenkins > Global Tool Configuration a different way. Rather than trying to get the zip installer from

https://repo.jenkins-ci.org
Try getting directly from https://repo.maven.apache.org.

Check https://repo.maven.apache.org/maven2/io/qameta/allure/allure-commandline/ to determin which version you need.

Then, write the url of the zip version you want to install, for example:

https://repo.maven.apache.org/maven2/io/qameta/allure/allure-commandline/2.13.5/allure-commandline-2.13.5.zip

And you'll be fine. The installer should look similar to the following:

Apply the changes, go back to the job and see the magic happening!

I hope this helps anyone!

Can we retry failed tests with testng? IRetryAnalyzer

gcalvoCR — Fri, 17 Jul 2020 22:42:40 +0000

Automation runs could fail for an endless number of reasons:

Flaky tests,
Running environment not steady,
Bugs,
Changes in selectors,
Framework problems,
And so on and so forth...

One of the aspects we definitely need to take into count is to investigate the root causes why the testcases fail, but in this post I want to bring up another topic.

If the team wants to rerun the failing tests under certain conditions, the different frameworks and programming languages usually offer diverse mechanisms that could let us re-run testcases programmatically.

Knowing that "Selenium-java" is still relevant and many people still use frameworks such as testng, I want to bring up the specific feature that testng offers, because it's a feature that's been available for quite a long time, and one I hadn't taken advantage of it, up to recently.

It's pretty straight forward, all what we have to do is to add the retryAnalyzer property to the test decorator and create the class that handles the logic for the retry.

Let's look at the following chunck of code as an example:

First we add the property.

@Test(retryAnalyzer = DynamicRetryAnalyzer.class)
    public void verifyWidgetOnPage()  {
        runTheTest();
    }

Then we create the class with the logic to retry your tests.


import org.testng.IRetryAnalyzer;
import org.testng.ITestResult;

public class DynamicRetryAnalyzer implements IRetryAnalyzer {

    int count = 1;
    int maxRetryCount=2;

    @Override
    public boolean retry(ITestResult iTestResult) {
        if(count<maxRetryCount){
            count++;
            return true;
        }
        return false;
    }
}

In this specific example the framework will retry a fail test an extra time no matter what (just if it fails). The example is kind of simple, but, I think we get the picture. If we need any specific logic to evaluate wether we should rerun a test or not, we do it by changing the logic inside the public boolean retry(ITestResult iTestResult) method.

The only thing we need to make sure is to implement the IRetryAnalyzer interface and override the retry(ITestResult iTestResult) method with the logic that we need to evaluate if we should rerun the test or not.

I hope this helps anyone!

"A test which fails is not always flaky!"

Alien symbols in Jenkins!

gcalvoCR — Tue, 26 May 2020 17:29:42 +0000

If you ever come accross some these "alien symbols" in Jenkins, specially on Windows:

And you want to make it up, the problem is the default encoding system and the solution is pretty straight forward:

Navigate to the directory where jenkins is installed. Usually C:\Program Files (x86)\Jenkins.
Once in the folder, open the jenkins.xml with Notepad, VSCode or any text editor.
Add --> -Dfile.encoding=UTF8 to the arguments.

The arguments should look something like this:

<arguments>-Xrs -Xmx256m -Dhudson.lifecycle=hudson.lifecycle.WindowsServiceLifecycle -Dfile.encoding=UTF8 -jar "%BASE%\jenkins.war" --httpPort=8080 --webroot="%BASE%\war"</arguments>

restart you Jenkins service and voilá!

After that, if you run the job/pipeline, you should see the console output showing the logs properly!

That's it, keep having fun with your CI/CD pipelines!

Callbacks in JavaScript. Let's break them down!

gcalvoCR — Fri, 17 Jan 2020 15:54:25 +0000

Callbacks are a simple concepts that many times confuse people when they are learning how to program.

Leaving aside that depending on the programming language the implementation could vary and there are synchronous callbacks (immediate) and asynchronous callbacks (may happen at a later time). Let's just focus on JavaScript.

Let's take the 3 following concepts:

First of all, the formal definition of a callback is

"is any executable code that is passed as an argument to other code that is expected to call back (execute) the argument at a given time"

In JavaScript you can assign functions to variables
JavaScript methods (functions) are first class objects, you can pass them around like variables

Knowing that, let's dive into an example to make it crystal clear.

Let's say that we define an execute method that receives a variable and a (callback) function.

function execute(px, callback){
    let x = callback(px)
    console.log(x);
}

I know, it's not necessary to instantiate the x

In this simple case, the callback function could be any function that has one parameter, so, let's define 2 more methods (functions) to test the execute method.

function squaredNumber(px){
    return px*px;
};

function numberPlusHundred(px){
    return px+100;
};

So, if we run:

execute(2, squaredNumber)

>>> The execute method will call squaredNumber(x) and the result would be 4

But if we run:

execute(2, numberPlusHundred)

>>> The execute method will call numberPlusHundred(x) and the result would be 102

It is as simple as that, using callback functions enable us to call different functions as needed making the programming even more flexible.

Final Thoughts

The given example could be way too simple but trust me, callbacks are used everywhere! To mention a couple of examples, Jquery and Ajax have them implemented in many methods. So, you should master the concept and take advantage of their benefits (Abstraction, Maintainability, Readability), and so on.

Any comments?

The challenges of QA Automation Engineer. Some things I have noticed!

gcalvoCR — Tue, 14 Jan 2020 20:09:49 +0000

We as QA Automation Engineers have lots of responsibilities, tasks and things to learn in our daily basis. We love solving problems, learning new cool stuff to apply to our projects, reporting bugs and off course automating everything. The job as a QA Automation Engineer is usually exciting and in many cases challenging.

Talking about challenges, there are some that I would like to point out.

1. Team communication and collaboration

There is still a gap between developers and the QA teams which we need to close. Usually, developers don't program taking in count automation needs, and it could be different. We should start thinking about working closer together. We could add more value to the projects by reporting issues earlier. Also, let developers know that, we could be that "safety net" for the programming processes, allowing them program with more confidence and less stress, just to mention one point.

I think there are many small "tweaks" that developers could add up to the various elements that help a lot the automating processes, such as, adding ids or unique names to the elements we need to automate. These minimum adjustments are nothing for developers and could have great impact on the automation.

2. Test case maintainability

With the products always changing and adding new features, test cases change all the time. If the application has many features, it could be translated into thousands of test cases which are tough to maintain. Therefore, we need to automate taking that in count. That means, making frameworks as flexible as possible, following best practices, allowing parallel testing, having a automation strategy, and so on...

3. Functional testing

Functional testing is great and that's usually the approach we take to assure quality, but we need to understand that not everything could be automated and not everything is functional testing.

On one side, we need to combine functional testing with sorts of testing. That's where other approaches such as visual testing could help us a lot, because it could make tests more reliable.

On the other side. Automation is not a replacement of manual testers. Automation should be used to take the repeated work away from manual testers, so that, they could use their full focus and strength in finding new testing scenarios and bugs.

Lastly, we should have an automation strategy...

4. Testing data

Automation usually needs data. In many cases, that means creating entire projects or modules devoted for that purpose.

This projects have many advantages, for example, they could avoid using the UI to create data making many test cases less flaky, they shorten the test duration, and make the test cases go directly to their point.

Therefore, anytime we need any specific information we should create it under the hood, but in doing so we need to pay attention to adapt the test cases to avoid creating unnecessary data, because we could be filling the databases of the testing environments with garbage which is usually annoying for everybody (especially developers) and it could cause some additional problems.

Conclusion

Test automation is an interesting and rewarding career. It certainly has its challenges. I just mentioned four but there are many more.

We should start communicating better, close gaps between colleagues, maintain test cases, have automation strategies, take advantage of different testing approaches and adapt our tests to our needs. In doing so, we will have many benefits, not only for us (QA teams), but for the organizations we work for as well.

Let's keep learning, improving, testing and automation our projects!