DEV Community: Rodrigo Matola

Stop using Appium (if you have access to the source code)

Rodrigo Matola — Fri, 27 Oct 2023 04:06:41 +0000

This text is to tell one reason that, if you have access to the source code of your mobile app, you should stop using Appium and use the native testing framework instead.

Context

Most of the mobile projects I worked on did not have a complete mobile CI pipeline due to the costs involved in building/renting a device farm. Only in 2 projects the mobile tests ran on CI: one the CI already existed and the other we built during the project, both running on a Mac Mini, and not in an automatic way.

What usually happened is that I coded my tests, using Appium with Ruby, and ran them on my own machine. The problem is that putting an Android emulator and an iOS simulator running at the same time practically means you can't use your computer for anything else.

In this situation, the tests only ran if I ran the tests.

In the project that gave the idea to write this text, the Devs did not do unit tests and the “it's ready, we just need to test it” was often said. So all responsibility for testing relied on me and the other QA.

As running the tests on the emulator and simulator at the same time “locks” the machine, it ended up that there was only automation code, but the tests were done manually.

The idea

One day, while writing testing code, I wasn't able to make Appium click or validate a field on iOS at all. Then I thought: “enough! This responsibility cannot be only mine!” That happened during the morning.

In the afternoon I began to look for how to automate UI with Swift and found this course from Test Automation University (TAU), from Aplitools. I spent this afternoon learning XCUITest. The course has 1h30min. I must have taken about 3 hours to finish.

The next day I automated a feature as an example and called the dev to talk. Since everything was already done in Ruby, it was just a matter of “translating” it into Swift.

What would be the idea: the idea is, since the tests will be at the same project, in the same repository, when the developer finishes the code, play the tests before push the code. If a test broke, we could look together at what was happening and fix the problem.

Even though I was still responsible for coding the tests, at least the tests would run, even with manual triggering. And I wouldn't be the only one responsible for that. I would no longer need to worry about automated scenarios.

After a while I was already being asked when it took me a while to merge a test PR! We achieved 73% coverage on iOS practically overnight!

Of course, I also looked for Espresso courses for Android. And I also found it at TAU.

Appium X Native Automation

Advantages of Appium

Several languages (Ruby, Java, Python…)
1 code, 2 platforms (or 3, if we consider web)
Test on the apk/app, which is what the customer will receive
Automation of “the entire device”, for example camera, gallery…
Easier to get material about E2E testing

I was reading that if you put a security tool, which I don't remember the name of, in the application, it is mandatory to test it on the apk/app, because directly in the code it is not reliable.

Disadvantages of Appium

“Segregated” QA
One responsible person
Additional integration required, as the code is in another repository
Slower (up to 8x)

Disadvantages in my context

It is necessary to build the apk/app on the QA machine
Run automation manually and locally, since we didn't have a CI
Free CIs do not include the diversity needed for mobile testing
“Impossible” to use the computer while running mobile tests

Remembering that these lists are based on my experience. For you it may be different.

Native automation advantages

“Dev” language
You “guarantee” that the tests will be run (we have to trust our team mates)
Split testing responsibility
More interaction and exchange between Devs and QAs
“Easier” to run locally

Disadvantages native automation

Learning and dealing with two (or three) different languages at the same time
Maintenance in two (or three) codes
Automation only in the context of the application. Camera and other interactions need to use UIAutomator, in case of Android.

Talking to other QAs who use this approach and reading some articles on the internet, the advantages outweigh the disadvantages. And I'm seeing it.

Getting started with native automation is just the beginning. In the near future I want to go deeper in the test pyramid and help with instrumented and unit testing.

And you, what do you think of the idea of migrating your tests to the development language? Comment here!

And share this post so more QAs see that it's cool to be closer to development.

And keep Testing Softly!

Who writes the BDD?

Rodrigo Matola — Sat, 11 Mar 2023 04:36:39 +0000

Being short and sweet, except for Dan North, who created BDD, I’d say

Nobody writes the BDD

What you write are stories, scenarios, cases, scripts, screenplays or any other name you like but BDD. If we translate the meaning of the abbreviation, Behavior Driven Development, you will see that it makes no sense to say “I will write the behavior-driven development”.

“BDD uses a story as the basic unit of functionality”*.

BDD is a practice. What you can do is describe this practice, not write it down. Making a comparison, it is the same as asking “who writes Scrum”? In this case we have two responsible, Jeff Sutherland and Ken Schwaber, who write the Scrum Guide. The rest practice Scrum just like they practice BDD.

"BDD is as much about the interactions between the various people in the project as it is about the outputs of the development process"*

*Quotes from Dan North’s article “What’s in a Story”.

BDD, Cucumber, Gherkin

It is very common to treat these three terms as if they were the same thing. Despite being closely linked, each term is independent and have “its own life”.

BDD

It is a software development practice that consists of writing application behavior before testing and development. I put tests first because BDD was a response to TDD (Test Driven Development).

BDD would be another version of TDD, with behavior-based tests written before development. Like TDD, BDD consists of 3 phases:

We start by collaboratively discovering the scope of behavior required by the story.
Once we agree with this behavior, we write the specification in business-readable language.
Finally, we automate the written specification to verify that the system actually behaves as expected.

Cucumber

"Cucumber is a tool that supports BDD.

Cucumber reads executable specifications written in plain text and validates that the software does what those specifications say. The specifications consists of multiple examples, or scenarios.

Each scenario is a list of steps for Cucumber to work through. Cucumber verifies that the software conforms with the specification and generates a report indicating success or failure for each scenario.

In order for Cucumber to understand the scenarios, they must follow some basic syntax rules, called Gherkin". (Copy and paste from What is Cucumber?)

Gherkin

Gherkin, or baby pickles, are tiny cucumbers… (just kidding!)

"Gherkin is a set of grammar rules that makes plain text structured enough for Cucumber to understand". These rules involve keywords that allow Cucumber to transform written text in natural language into steps for automation. Given, When And Then are some of these keywords.

Although Given, When And Then are officially part of the language used in BDD, Dan North says "This is not mandatory — you can use a different story format and still be doing BDD — but I am presenting it here because it has been proven to work on many projects of all shapes and sizes".

In short:

BDD: software development practice
Cucumber: tool that converts text into steps for automation
Gherkin: language used by Cucumber.

As said at the beginning of this section, the three terms are most often used together, but they are completely independent entities.

Some frameworks do not use Given, When And Then, such as Gauge, Robot Framework and even Cucumber itself! I can talk about this in another article.

You don't do BDD

I’m sure many of you, like me, learned BDD together with Cucumber as a tool for automating tests. I just say that

If you only use Cucumber to automate tests, you don’t do BDD

There’s nothing wrong with this practice, we just can’t call it BDD. As the name implies, BDD is a development technique, not a testing technique. If you use it just for testing, you do BDT (Behaviour Driven Testing).

Until now, only in one team we almost did BDD…

And you, do you do BDD or BDT?

References

Introducing BDD, Dan North
What’s in a Story?, Dan North
What is JBehave?, JBehave.org
Behaviour-Driven Development, Cucumber Docs
Myths about BDD, Cucumber Docs
BDD with Cucumber, Cucumber School
Why is BDD one of the most misunderstood concepts?, Specflow.org
Myths and Challenges of Behaviour Driven Development, Pankaj Nakhat (SlideShare)
Behavior Driven Testing in Automated Testing, Alex Jones in Devops Zone

Number of Devs X Number of QAs: Why do I think it's an ineffective measure?

Rodrigo Matola — Mon, 21 Nov 2022 02:14:58 +0000

Once, after saying that I had a lot of things to test, I was asked how many developers per QA were in my old job, since I always referred to it as having “ideal teams”.

I replied that they were 4 to 6 devs. It was usually 2 Android and 2 iOS developers, and some teams had 2 more for the backend. I received a comment: “hey, it's the same amount we have here. Why do you complain about a lot of work here and there it was ideal?”. At the time I was asked this question, we had 5 devs.

Task team

The difference was that in the reference teams, all members were working on the same functionality. For example a login. In planning, the designer presented the idea of a screen. If it was too complex, we would already adapt it to be able to deliver in a Sprint.

Frontend developers were usually split with one person laying out the screens and the other doing the functions. Unit/instrumented tests were also split. In the planning, the contract with the API was also defined and mocks were delivered before the services, for the frontend devs to be able to start development. Each one was responsible for testing their code. If unit tests were covered, QA didn't have to worry about these tests.

In the “current” team each dev takes an item from the backlog. Taking a bank app as an example, and 4 devs, login, transfer, pay and deposit will be developed in a Sprint. Also, the backend is developed by another team.

To get worse, NO unit testing was done. The phrase “it's ready, test it” or “it's ready, just need to test” always happened. That is, all responsibility for the QA. It was common to receive a “done” where the main action button did not work. Or something that was working break with the merge of another functionality. It was practically impossible to make a decent regression and bugs always went into production. Same for the APIs.

So, in addition to testing the front, it was necessary to test the back. Almost always the frontend devs had to do some workarounds to fit the backend's responses, which usually caused errors if the API responded differently.

In this context, I've heard "you haven't tested this, why?" or “I'm sorry, but your tests are shallow”. My analogy was “it's impossible to deep dive into 4 or 5 pools at the same time. You can only take the leaves off the surface.”

Correct metric: features / QA

With that, the metric I consider correct is features per QA (or tasks/QA). So we have a measure of how much a QA is being “overloaded”. The ideal number will vary for each QA, with the size/complexity of the functionality and for each team.

I would follow the “two is company, three is a crowd”, for a two-week Sprint. Before I want to make it clear that the quote is valid for “total QAs”, that is, those who care about the end-to-end quality in all layers, and not “only do tests”.

Two is company when talking about completely new functionalities, where the backend and frontend are being developed from scratch. In this case, even though it's just one feature, you're looking at two different systems. And I'm not including automation. If we include automation, perhaps one functionality is enough.

The team also has a big influence on metrics. If everyone cares about quality, QAs can play their role much better. On the other hand, task teams, like the one described above, have a high chance of bugs being produced, and not resolved.

Find your number

Over time, in the sprint planning you will be able to measure the size of the task and its complexity. Align the priorities with the PO and choose the two most critical features for a “perfect delivery”, features that you will dive deep into and “clean up” everything possible.

I recommend taking a look at PRISMA (Product RISk MAnagement) technique, from Erik van Veenendaal's. And always try to include automation within the Sprint.

Finally, just because you chose two main features doesn't mean you'll ignore everything else. Unforeseen events happen, bugs in production arise, new requirements are discovered. Focus on the main features but also take a look at the application as a whole.

And you, do you know leaders who use this Devs / QA metric? Tell me how it is working!

NEVER trust in unattended data

Rodrigo Matola — Sun, 13 Nov 2022 14:34:25 +0000

Cover image by Jen Theodore on Unsplash

I will start with the phrase

"Against data there are no arguments"

and show in this text that there are arguments. And many…

This text can be considered an addition to "Automation Metrics: some problems".

Contextualizing

Data and metrics, depending what company, are difficult information to have access. They are often in the hands of another area (which is not IT and development) and it may be bureaucratic to gain access to Google Analytics, for example. And you may not get it.

This situation, for us QAs, is very bad, because we are left without foundations on what to prioritize in tests and automation, being totally in the "guess".

However, sometimes (always?) loose data comes to your attention, such as average age of customers, operating systems and browsers used, average browsing time on the site/application etc.

Since that's all you have, you end up making your testing strategy based on them. But is the information that came to you complete? Or worse, correct?

The examples I will use are real, but with adaptations to fit the text.

Android or iOS?

In a project that I worked on, we were doing very superficial tests on iOS, only happy paths. That's because the information we had was that about 70% of customers use Android. New features were prioritized for Android. Bugs on iOS, depending on the severity, would find a country house to vacation and not be disturbed.

I think 30% is a very large number to ignore, but this strategy was agreed with the team and the business area.

In a certain review, after me and some devs wanted more data, a dev brought some numbers he got from Firebase.

We found that Android was used by 90% (not 70%), which gave me a relief to know that our strategy was even more right. Relief that only lasted until the next slide.

This slide showed that Android customers spent an average time of 1.5 minutes on the app, while iOS customers spent around 7-10 minutes (I can't remember the exact time). This information immediately conflicted with the previous one in my head! A minute and a half could mean that people just installed the app to find out what it was about and never used it again! Or used it sporadically.

The time of 7 to 10 minutes was the time it took to make a complete flow in the app, that is, our focus maybe should be iOS and not Android! Was our whole strategy wrong?

I went to another team in the next Sprint as the apps were deprioritized for a while. I don't have any more information about the metrics...

Time of section

Once we had to set a session time for the person to be logged into the application. This was an Information Security (IS) imposition.

IS wanted us to set 3 minutes of session time. This time was enough for a query in the app. We had that same 3 minute value as an average time extracted from Analytics. We implement this value.

Few days (or weeks) later we started to receive complaints that the app was logging off by itself while browsing. We noticed that for some flows, 3 minutes was the time the person would spend if they made the flow directly and without errors, that is, almost like an automation. Of course, ordinary people were taking longer than that.

A large part of this complaint came when we launched a new paid functionality, previously restricted to a few customers, to basically the entire customer base. As new customers were unable to complete the registration process, they began to complain. Imagine the money lost during that time because of an incorrectly used metric.

Average alone means nothing

The following example was adapted from UOL Educação (in Portuguese).

Your street has several children and you want to develop a product focused on these children. You go to the street houses to ask the age of the children and set up the following table:

Ages: [9, 9, 9, 1, 1, 1]; 
Average = 5

Then you make your products aimed at 5 year olds. Do you agree that 1-year-olds and 9-year-olds are in very different stages of development than 5-year-olds? You do not sell even one unit of your product.

"Average must be accompanied by at least the standard deviation"

If you calculate the standard deviation of this sample, you will find the value of 4 for more or less, that is, despite the mean being 5, you can have children aged 5+4=9 and 5-4=1, exactly as in your sample!

Another example, which is the time taken for a hospital discharge process, was taken from Minitab.

In the two graphs above, the average time is 35 minutes (dashed line), but the standard deviations are 6 and 20, respectively. That is, the average in the first case is more reliable than in the second.

In the first case you would wait between 29 and 41 minutes (30-40 minutes, rounding up). But in the second, you would wait from 15 to 55 minutes! Imagine you asking "how long is the wait? From 15 to 55 minutes", would you consider it a reliable measure?

Use other central tendency measurements

Measures of central tendency indicate a value that best represents the entire data set, that is, they give the tendency of concentration of observed values. (Ref. Introdução à Estatística Econômica, UAB/FURG. In Portuguese)

In addition to the average, when doing a data analysis, also take a look at the

median: value that is in the middle of the sequence, when the data are arranged in ascending order;
mode: value or values that occur most frequently.

See the dataset below, taken from <https://matematicabasica.net/media-moda-e-mediana/>

You can see that the values are much dispersed, and this is confirmed by the measurements:

average: 42.1
deviation: 23.2
median: 45.0
mode: 20.0 and 50 (bimodal)

Imagine that these were the ages of the people using your app and you need to focus your adds on an age group as your budget is low. What age would you focus on?

With that alone, can I make an assertive analysis?

No! For example, do our customers use Android or iOS? This is a hypothesis to be tested, so it is necessary to do a hypothesis test. It is also necessary to define the confidence interval.

The data must be treated to verify its consistency, if it has outliers, which distribution the data obey (normal, chi-square, etc.)

Anyway, did you realize that just an average is basically useless?

Conclusion

The purpose of this text is to warn that you cannot make decisions based on a single source of information, and not that you need to make fancy calculations (such as hypothesis testing) or have statistical training to do so.

On the news, you should go after more information to confirm the veracity. The same with data. Not that you need to consult another data source, as you will often have only one source, but look for other interpretations and analysis for the same dataset before making a decision.

How I automate tests (or try to) inside Sprint

Rodrigo Matola — Sun, 13 Nov 2022 14:28:56 +0000

This is an old post. I don't use Cucumber anymore, but the ideas still the same.

Attending Meetups and talking to QAs from other companies, the difficulty of implementing test automation within Sprint always come as subject. So I decide to tell how was (and is going) my experience with test automation. Note that this is my recipe and worked for the projects that I passed. Consider if it fits in your case, ok?

In order for the method work, some backgrounds are needed, based on conversations and readings about test automation and my experience.

Company

The most successful cases cited in the conversations, the company (or at least some managements positions) take the struggle to run agility regardless of the framework (whether Scrum, Kanban or other). In these cases, if the team fail, they were encouraged to try again, and not punished for mistakes.

This struggle also goes to the stakeholder that wants to “waterfall” the process. If your company complies with stakeholder’s decisions on how the team should work, to not lose the sale, the chances of failure are high.

Team

We will consider here team as a group of people working to hit a common goal that adds value to the product, all together.

The team must share information, ideas, concerns, success and failure, working as a single entity.

A team differs from a work group the way people behave. In a work group, each one does their own job without thinking about how it can affect the others. Has the Sprint failed? “I deliver my work on time, I have nothing to do with it.” Or worse: “who did not get the job done was that person there,” pointing the finger.

I’ll put one more category, the “team-ish”. The team-ish knows how to work in group, but only do what is written to do. They may even think of improvements, but if the goal was achieved, why do more?

If you’re in a work group, you’ll hardly be able to automate during Sprint. In other cases, the likelihood is high. And if you’re in a team-ish, why not try to make it a team? Tell your ideas of integration in the retrospective or call the team to chat after a daily, since you will be together.

Imagination

To automate a feature that does not exist takes a little imagination. If you already have the drawing of the screens becomes easier, but if you haven't, imagine the flow. Your code, at first place, will automate features like filling a field or the click of a button regardless of its position on the screen, color or shape.

“But I don't know IDs !!!”, you say. Two solutions I use:

Leave the field blank or with some indication to complete later, or
Create a variable that allows key-value construction (hash, hashmap, or dict, for example). Fill in the field with variable_name [key]. Then just complete the second member of the equality with the values that you are discovering.

Using this method I have already completed my code along with Dev! So, just map the IDs and make small adjustments that the functionality was already automated.

My flow

It all starts at the planning meeting, where we decide the sprint backlog. Let’s assume that the purpose of the sprint is to perform the login (always login as example …).

Write down all the possible scenarios for a login. Before starting to automate all, define which will be unit / instrumented, which will be services / APIs, and which will be the front.

Divide the responsibilities and automate first all the “happy paths”, those in which everything will be done perfectly. There is no point in a feature that does not perform its function if the user does everything right.

Going back, write all the possible login scenarios successfully, for example:

#login.feature

Feature: Successfull login

Background:
   Given I am on the login screen

Scenario: Successful login using username
   When I log in using "username"
   Then I should be directed to the main page

Scenario: Successful login using email
   When I log in using "email"
   Then I should be directed to the main page

Scenario: Successful login using ID
   When I log in using "ID"
   Then I should be directed to the main page

Write down all that your system will support.

Implement the scenarios

Here comes the imagination. Scenarios will be implemented only with login screen drawing.

My steps file, without the implementation, looks like this:

#login.steps.rb

Given ("I am on the login screen") do
   pending # write code here that turns the phrase above into concrete actions
end

When ("I log in using {string}") do | string |
   pending # write code here that turns the phrase above into concrete actions
end

Then ("I should be directed to the main page") do
   pending # write code here that turns the phrase above into concrete actions
end

Note that even though my feature file has several scenarios, there is only three steps to implement. When writing anything in your code it is very important think about reuse. After implementing “with my imagination”, the steps will look like this:

#login.steps.rb

Given ("I am on the login screen") do
  @login_page = LoginPage.new
  @login_page.load
end

When ("I log in using {string}") do | user |
  @login_page.user_field.set USER[user]
  @login_page.password_field.set PASSWORD[user]
  @login_page.btn_login.click
end

Then ("I should be directed to the main page") do
  current_page_is?(NAMES['current_page'])
end

And the page objects, variables, and methods files, using SitePrism and Capybara:

#login.po.rb

class LoginPage < SitePrism::Page
  set_url ""

  element :user_field, ""
  element :password_field, ""
  element :btn_login, ""
end

#methods.rb

def current_page_is?(page)
  page.has_title? page
end

#variables.rb

NAMES = {
  'main_page' => ''  
}

USERS = {
  'username' => '',
  'email' => '',
  'ID' => ''
}

PASSWORDS = {
  'username' => '',
  'email' => '',
  'ID' => ''
}

Now just wait for the login screen be done, complete the code and the feature is already automated! Said no one ever …

Even if you replace everything right, the chances that you make some mistake in the first (second, third …) run is huge. The above example serve only as a basis. Try to fix the errors as they appear in log. Even with these errors, it is easier and quicker to make adjustments than waiting for the functionality to be fully ready to write the code. Errors will decrease over time.

Also, it’s not just because you’ve finished a feature that will sit and wait. In an agile team, there is always something to do. Talk with the responsible for DevOps and see how the CI / CD pipeline is going, see how is going the application development with the developers, talk about the backlog and usability. Interact with the team, give suggestions for improvements, point problems (already with solutions, if possible). You are responsible for product quality, not just automate testing.

Don't forget the exploratory manual test

Always take a time to use your product. Only this way you will have the complete knowledge of how it is working. Don't automate only the requirements described in the stories. Many issues and even new automations will emerge as you use the application. Automated and manual tests are complementary, not exclusive.

I would dare to say that today, with the great demand for automation, QAs should also “step back” a little and do manual testing.

I hope you can get the general idea of this text and apply in the context of your company.

And don't forget to comment here how the experience is going! And if you have a question or want to share your experience, leave a comment below!

Automation metrics: some problems

Rodrigo Matola — Mon, 29 Aug 2022 04:26:40 +0000

“What is not measured is not managed”

This phrase is one of the most used when justifying the implementation of metrics in a project. another is

“Against data there are no arguments”

A big problem I see in these sentences is that metrics, without interpretation, are just numbers. And numbers alone have no obligation to reality.

A great example to illustrate what I mean is the Spurious Correlations website. This site comes to absurd conclusions, as the increase in consumption of mozzarella is correlated with Civil Engineering doctorates awarded, or that decreasing the consumption of margarine, also decreases the number of divorces in Maine.

What I want to say in this introduction is that numbers, metrics and even statistics can lead us to very wrong conclusions about our process/product.

Now let's get to the main subject.

The 3 numbers

The 3 numbers are not a consolidated technique for quality metrics. The 3 numbers is a metric that, all of a sudden, a stakeholder asked all QAs to report. These 3 numbers were intended to inform managers about the evolution of test automation:

number of mapped scenarios
number of automated scenarios
automated scenarios in the last week

These numbers were a weekly report that we should send, as the company is "goal oriented" and each manager must achieve one.

Let's now enumerate some problems I see in these numbers.

Only numbers don't mean anything

Numbers are just collected data. What you look for when collecting data is information.

Information is the ordered and organized data so that it conveys a comprehensive message within a given context. A set of data that conveys knowledge is information.

That is,

information = data + context + interpretation

A simple example: I pay $2000 in rent. With this data alone, it is impossible to say whether I am paying too much or not.

If I live in a studio in Osasco, a São Paulo nearby city, $2000 is an absurdly high value. If you go to a 3 bedroom apartment in Moema, it's basically for free.

For QAs, especially if part of an agile team, the Sprint may have required tasks other than automation:

make API mocks due to instability of the test environments or due to the difficulty of getting mass data;
refactor the test code, because there were some modifications in a functionality or correcting flaky scenarios
work on the continuous integration pipeline, or helping with it;
it was a “manual” Sprint due to the urgency to launch a feature, to not miss the “time to market”, and automation would take more time than needed to launch

These are just a few examples where the 3 numbers, without context, could lead to misinterpretations of why automation didn't progress or was slower in a Sprint.

It will never be 100% automated

When we talk about test scenarios, we usually mean Gherkin and BDD. I argue that, as the name is Behavior Driven Development, ALL behaviors in the software must be described, not just those that will be automated.

As a result, scenarios that can only be done manually, or that the automation and maintenance effort is very high, will not be included in the statistics. An example is putting an iPhone simulator in airplane mode, to test how the application responds to lack of internet, or turning on bluetooth in an Android emulator.

In addition, some scenarios, even containing automation code, will not run. For example a registration feature.

Scenario: Enter the registration area
Given the user is on Home page
When access the registration area
Then the form should be on screen

and

Scenario: Successfully register
Given the user is on Home page
When the user make the registration
Then a success message should appear

The first scenario can be discarded and the second needs to run, because in order to register it is necessary to display the form first.

In this case, we will have two scenarios mapped, but the report will contain only one executed. If we extrapolate, we will only have 50% of the tests running automatically, but 100% of the scenarios covered.

To achieve the goals, we will hire "automators"

In an agile team, having people to perform only one task is not "healthy". In this case, having QAs and "automators" can (will?) generate dysfunctions.

For automators, a detachment from quality: “I'm here just to automate”. For the manual testers, a feeling “if you automate everything I could lose my job”, or “I also want to automate, why don't you teach us?”, causing demotivation.

At some point, this will create cliques, decreasing interaction between people and even generating hard feelings between them. As a result, quality is no longer a priority.

To put the above into context, the company, which only had manual QAs, hired an automation consultancy company to speed up the process.

Conclusion

My objective with this article is to check if we are using only data, or if we are using information to make decisions.

As we can't just consider the rental price for renting a house, we also shouldn't rely on just one or “three numbers” to measure an evolution.

Tip: managers, participate more in the development cycle. Get close to people, talk, find out what each person is doing and developing. They may have other even better numbers.

"Tell me how you measure me and I'll tell you how I'll behave"

This sentence fits the situation very well. But that discussion is for another text!

And you? Do you have a story of imposed metrics for your team? Comment here!

Ruby: "the best" language for general automation

Rodrigo Matola — Thu, 12 May 2022 22:42:19 +0000

Before you say "there is no better programming language, each one has its purpose", I know it. That's why "the best" is in quotes in the title.

But in this text I want to show 3 situations in which Ruby was the best choice, even though I didn't want to use it in two. With Ruby, I did in hours what I couldn't do in days (2 to 3) in another language (Shell Script, in this case).

The three situations that I will talk about here are:

Continuous compliance
GitHub Automations
CLI Test Automation (Command Line Tools)

But before getting into these topics,

Why Ruby?

Its creator, Yukihiro “Matz” Matsumoto, combined parts of his favorite languages (Perl, Smalltalk, Eiffel, Ada, and Lisp) to form a new language that balanced functional programming with imperative programming. He said, "I wanted a scripting language that was more powerful than Perl and more object-oriented than Python." (From https://www.ruby-lang.org/en/about/)

Ruby is highly portable: it is primarily developed on GNU/Linux, but works on many types of UNIX, macOS, Windows, DOS, BeOS, OS/2, etc. It comes by default on many Linux distributions and on macOS.

Ruby is not only completely free, but also free to use, copy, modify and distribute.

It is considered one of the best languages for test automation and one of the pioneers in some frameworks, such as RSpec and Cucumber. But unfortunately its popularity is decreasing.

Where is Ruby used?

Below, only 3 examples where Ruby is used:

GitHub
homebrew
cocoapods

My experience with Ruby

Basically my entire career as a QA was automating in Ruby. Only recently with the popularization of Cypress for web testing, I migrated to JavaScript as the main language.

Now let's get to the main subject, which is where I'm using Ruby even though it wasn't my first choice.

Continuous compliance

I didn't even know it existed until I watched TAU's Continuous Compliance course. This course opened my mind to a sea of possibilities for automating checks.

The course uses Chef Inspec, an open source Ruby DSL. I made a POC with this tool to automatically check repositories on GitHub, checks like if it contains a gitignore consistent with the language used, if node_modules is not present, etc.

The problem is that Chef Inspec must be installed every time a workflow runs, in addition to having to run a specific command, the repository being in a specific folder, having to look in the documentation for the command to perform a check, which is sometimes limited …

After reading the documentation and knowing the possibilities for use, I ended up abandoning the tool to use pure Ruby. If the tool made in Ruby does it, obviously pure Ruby will do it too, and much more, because there will be actions not implemented in the DSL (this is one of the reasons I'm a bit against DSLs, but that's for another text).

GitHub Automations

Probably everyone here has already created a repository on GitHub, but I'm sure if you don't use GitHub for work, you've never bothered to secure a branch, for example.

Some settings are essential when working with GitHub, such as protecting the develop and main branches with some rules:

Do not accept direct push
Merge with pull request only
Require signed commits etc

These settings are a little boring to make. When creating a repository, it can take some time clicking on (checking) checkboxes. This time can be reduced by scripts for automating the creation of repositories through the GitHub API.

As I wanted these scripts to be able to run on both Linux and macOS (sorry Windows. Hi WSL), I started to develop them in Shell Script, since it is a native language of both systems.

But when it came time to do some checks before an action, Shell Script became extremely complicated. And what is the other language that comes with Linux and macOS? That's right, Ruby.

I used Ruby to do some checks like if the repository exists, list repositories and call existing sh files.

I thought about using JavaScript, as we have more people who develop in that language on the team, but Ruby was accepted as the best choice.

And to make it even more "system compatible", this is inside a VSCode devcontainer, with the Ruby Docker image, making everything configured with one click.

CLI Test Automation

We were developing a command line application. I had never worked with this and had no idea how to automate CLI tests. Obviously I went to ask Google.

One of the results was how to test CLI developed in Rust, just what I needed! Unfortunately, for lower-level testing, the code would need to be refactored to be testable. And I don't have the knowledge in Rust for that.

Fortunately this and several other results told how to test, which is simply typing a command and checking what comes out in the terminal. So I would just do e2e for now.

The tool that was most indicated was Bats, made in Shell Script. But I couldn't make some simple things work like assigning variables to use them within the tests. The documentation itself said that it was a little complicated to split the tests into different files. Imagine the mess that would be as the tests grew.

Another tool I found was Aruba, written in Ruby, but the documentation is practically non-existent and it recommends using it with Cucumber. Even the examples inside are in a features folder. I refuse to use Cucumber…

Then came the same thought from Chef Inspec: if it's done in Ruby, I can do it in Ruby. And it's working beautifully with Ruby, RSpec and the Open3 module for handling terminal commands.

Conclusion

Whenever I go looking for a way to automate something, if there isn't a widely used framework like Cypress, Robot Framework, Skuli etc, and there is some mention of a framework made in Ruby, I won't even look at the others. Ruby is the choice!

Where to learn Ruby?

I learned from Codecademy and, of course, from Google too. I recommend Codecademy for any language. Online hands on and straight to the point! Basic courses are free.

RubyGuides also has a lot of material and helped me a lot. It also has a newsletter that you receive content about Ruby directly in your email. That's great for me!

In the language official documentation, there are other links to learn Ruby.

Ready, Accepted, Done! Acceptance criteria and definitions of ready and done.

Rodrigo Matola — Tue, 08 Mar 2022 21:55:25 +0000

How to know if a task is ready to be started or done to be delivered? A chocolate cake for example, for it to be ready, does it need topping or not?

To align expectations, there are three agreements you should make before starting to develop a product:

Acceptance or Acceptance Criteria (AC),
Definition of Ready (DOR),
Definition of Done (DOD).

In this text I will talk about these three agreements using analogies outside of IT and also examples within software development.

Using these three agreements within a development team is extremely important so that, when a task is delivered, you don't hear from stakeholders "but that's not how I asked you to do it", "how is this ready if this is missing?", or the team "you didn't give me what I asked for, so I couldn't finish it".

Acceptance Criteria (AC)

Stakeholder: I want a chocolate cake in two days.

Dev: Right. With or without topping?

Stakeholder: With topping and filling, but I'm celiac so I need it with rice flour.

Dev: Ok. What size?

Stakeholder: hmm... just for family. Me, another adult and two children.

Dev: I need you to be more specific. Give me exact measurements.

Stakeholder: I want the cake in a 20cm pudding mold.

Dev: Topping with milk or dark chocolate

Stakeholder: 70% dark chocolate

Dev: Any specific brands?

Stakeholder: No. I just don't want with hydrogenated fat

Dev: Anything else?

Stakeholder: No. That 's it.

Dev: Check with me, please:

Chocolate Cake
Made with 20cm pudding mold
70% dark chocolate, non-hydrogenated
Rice flour instead of wheat

Stakeholder: Checked!

Dev: Let's define the payment?

In this example, we define the acceptance criteria for exactly requested cake to be developed.

In an IT context, this list contains the requirements for a feature to be developed as expected.

Definitions

Criteria required for a given task* to be considered completed
Differ for each task
Answer the questions:
- Are we building the right thing?
- Did we build it right?
Must (need?) be written before start development

*task can be story, subtask, epic…

Example

An endpoint must return the following status codes:

200 success
401 unauthorized
404 not found
408 timeout
500 server error

So the task cannot be given as completed if it doesn't return at least these statuses. Can it return other statuses besides these? Yes, it can, but it is not recommended to make increments without being aligned with the business party and the rest of the team. Modifications must be added/removed from the acceptance criteria.

Definition of Ready (DOR)

Dev: To start development, I need you to transfer to my account the value of the ingredients or you can have them delivered directly here. Without the ingredients I can't start the development.

Stakeholder: I will order now. Hmm… it says here that delivery will be in two days. Can you do it for the same day?

Dev: I can't. I need at least one day. So the ingredients must be provided by 4 pm the day before you need the cake.

In our story, the DOR for cake development is that the ingredients must be available by 4pm the day before the cake is delivered. Coming back to IT,

Definition

DOR are prerequisites for a backlog to be considered ready to start development.

Examples

Do we have all the access we need?
Do we have the hardware and software needed for development?
Do we have written acceptance criteria?
Do we have a wireframe, even hand-drawn? (for front end)

Explaining the examples:

If the task is to configure Amazon's S3 bucket, is it worth including this task without the responsible person having the necessary access to perform it?
Will you be able to start iOS development without having a Mac with XCode installed? It will be hard…
Are you going to start developing without knowing what should be developed? You can, but definitely you'll have to rewrite somethings
Same as item 3 for the frontend

Definition of Done (DOD)

Dev: Your cake is ready. You can come get it.

Stakeholder: I thought you would deliver it here.

Dev: We forgot to define it…

Although the cake is ready meeting all the acceptance criteria, can we close this task if the cake was not delivered? In this scenario, we have at least two possibilities for defining done:

cake done with stakeholder going to get it
cake done being delivered to a certain address

Definition

DOD is the criteria required for ALL tasks to be considered completed
Task done = AC + DOD
It's the same for everything in a given context
Context can be team, project and/or the entire company
It is an agreement for all people to follow in a context

Examples

Is it pushed (git push)?
Is it on staging?
Is there error treatment?
Was it functionally tested?
Was it security/pentested?

Explaining the examples:

Is a code that is only on your computer good for anything?
If a site is not in an environment that can be validated by others outside the team, is it ready? (May be, depending on team lineup)
Let's assume that a frontend consumes the API described in the acceptance criteria example, if the API returns "201 no content" or "206 partial content", how will the front behave? (I've seen infinite loadings on the web and apps crashing in cell phones)
It's done, just need to test! (Really? No comments.)
In this time when ransomware and data leak are happening a lot, isn't it worth having it in your DOD?

Note that done does not necessarily mean being in production. Done will mean different things in different teams.

To think:

If a task/feature is not able to deliver value, is it done?

So, does your team already practice these agreements? If not, shall you start?

Azure, CircleCI, GitHub, Gitlab: a POC

Rodrigo Matola — Fri, 07 Jan 2022 03:01:38 +0000

A limitation in GitHub Actions, despite being completely workable, made us question whether it would be better to change our CI tool.

The tech lead was very experienced in Microsoft tools, so he suggested Azure DevOps. I also like Azure a lot, but I have* a crush on Gitlab. The team's "DevOps"** had experience with CircleCI and suggested it.

In order not to have an unecessary discussion and not decide based only on documentation (we did this once and I don't know if it was the best choice, but it working very well), I offered to do a POC (Proof of Concept) with the CIs mentioned.

As we were already running the website build and tests on GitHub Actions, it was just "translating" GitHub's yml to the yml of other CIs. Said no one ever…

*had. I explain below

**Just to give another name than developer.

CircleCI

I started with CircleCI because one of the things we talked about was to keep the code on GitHub. That way we wouldn't need to do migrations, although that's pretty easy.

From the documentation, I was delighted with CircleCI. The "Orbs", which are like GitHub Actions, the possibility to choose machines with up to 128 GB of RAM, the possibility to choose graphics acceleration. I almost closed the deal with CircleCI here. But then came the construction of the pipeline…

My experience with CircleCI is only from studies. So my pipelines in CircleCI are pretty simple.

When I translated the GitHub Action to CircleCI yaml, it was the CI that took me the longest time to start. It took me a long time to understand the structure of the workflow, but that might have been a limitation of myself.

One point I didn't like how were the examples in the documentation. They only show simple structures. To do what I had in mind, I had to search the internet. I wasted a lot of time.

Another setback is that many of CircleCI's (all?) integrations need permission from the company, which is annoying and can delay development.

But what was knockout for me on CircleCI was the difficulty of running a localhost on it. I searched on the internet and saw that this is really a problem. There were other solutions, but I found it very complicated for something that is straightforward in other CIs.

CircleCI Pros*

Caching dependencies: If your project uses Yarn, Bundler, or Pip, for example, the dependencies downloaded during a job can be cached for later use rather than being re-downloaded on every build. The other CIs have it as well.
Docker Layer Caching (paid): save Docker image layers created within your jobs rather than rebuilding the entire image every time, saving building time. Has no effect on Docker images used as build containers. Is only useful when creating your own Docker image
Machines up to 128GB RAM (Windows), versus 7GB on GitHub. You can choose GPUs as well
Orbs vc Actions: orbs are just packaged, reusable YAML. Actions are written for execution inside a Docker container or coded as individual steps using JavaScript. So anyone can code Orbs, not just JS devs.
Can run jobs locally using the CircleCI Local CLI
Rerun from start or from failed job
A YAML validator, on CI and locally with CLI

Cons

Need authorizations to connect with GitHub and to do some other tasks
Need to config the virtual environment (this could be an advantage). GitHub Actions provides a "ready-to-go” operational system.
Some issues running a localhost server. I considered it an eliminatory issue
No built-in way to continue after failing tests, but display the test failed.

My opinion

YAML a little complicated to configure.
Lots of permissions to ask
Doc examples too simple. Had to google to find more complex examples

*The pros and cons were based in features that were useful at that time

Azure DevOps

Azure DevOps was the first CI I actually used, I mean, it was the first one that I built a pipeline from scratch (after Jenkins on a Mac Mini). This was possible because it has excellent documentation, with several examples.

I was able to do everything I wanted on it, so it quickly became one of my favourites.

A big advantage of Azure over the others (I don't remember if I needed to do this in Gitlab), is clone other repositories within the same domain into your pipeline. It was just a git clone like you do on your computer. On GitHub, for example, you need a PAT (Personal Access Token) and use their action/checkout.

Azure native tool for test reporting, to me, is the prettiest of all these CIs.

Another very positive point for Azure DevOps is the price. Free for teams smaller than 5 people and $6 per person for larger teams. Currently, it is necessary to ask for an authorization to use the pipeline, which can take a few days to be approved.

Azure DevOps Pros

Node JS: able to run --only=dev or --only=prod to speed up the dependencies install (don’t know in other package managers)
Good documentation
Easy to checkout other repositories from the same company
Rerun failed jobs
As GitHub, have a “whole” OS image, like ubuntu-latest not just tools like node:latest
Allows caching
Pricing

Cons

Just one YAML per pipeline. You can have more than one, but need to specify manually which one to run.

Gitlab CI

Gitlab has always been my favourite tool. I was enchanted with it from the first time I used it. I don't remember if at that time other tools had the same functionalities as Gitlab, but my first use and the team's choice to use Gitlab was because it already had a built-in Wiki.

Other native Gitlab tools I used were Gitlab Pages and Gitlab Runner, meaning Gitlab always had everything I needed. Also, it saves the files generated by the pipelines for quite long time, so I didn't need to add a step in yaml to save the test results somewhere else to have a test history.

Not to mention that Gitlab also has excellent documentation, and since I already had experience with Azure DevOps, it was even easier to understand.

The only bad thing about Gitlab, for me, is that it doesn't have the separation of jobs like other CIs. The entire log is on a single terminal. If your pipeline is big, it becomes a little confusing to find a broken test, for example.

But what was a knockout for Gitlab was the site's build time. While we other CIs stayed in at most 20 seconds, in Gitlab it was a minute! And I have no idea why…

Gitlab CI Pros

Static Application Security Testing (SAST - Free): Analyze your source code for known security vulnerabilities.
Some built in tests, like SAST mentioned above
Gitlab Pages (better than GitHub, to me)
Good documentation
Allow cache between pipelines and jobs. Specify a list of files and directories paths that are in the local working copy, e.g. node_modules

Cons

Only "raw images" like CircleCI
Too slow to build the website
No visual division between jobs

GitHub Actions (GHA)

Finally, I'm going to talk about the tool we were already using.

One of the things I most liked about GHA is that it has full operational systems, with several tools already installed. This practically makes me program the pipeline as if I were on my own machine, with no additional configuration.

The Actions, most of them, are very well documented and simple to use, so there is no need to search the internet for examples, only in specific cases or if you want to see other examples of use.

An advantage of GitHub over other CIs is that it allows you to have many different workflow files. This is great for organizing workflows. For example, you can have a file just to run the tests, another one just to deploy in staging, another one to deploy in production etc. Triggers are set up on each file and fired automatically if the condition is met. A disavantage is to call a workflow inside another workflow. Is not straightfoward and a little hard to do.

Azure DevOps also allows you to have multiple yaml files, but you have to define only one for each pipeline. I didn't find this possibility in other CIs.

Another thing that makes GitHub stand out are its APIs, Rest and GraphQL, and its CLI (Command Line Interface), which allow you to do almost everything from the command line and, consequently, automate it all!

Summary

CircleCI was the one that took me more time to start the job. I liked the separation of jobs and workflows, but generated too many errors before the first green pipeline.
Azure DevOps (together with GitHub Actions) was the most straightforward to begin
Gitlab I had to do some workarounds to deal with the lag of website build, because the pipeline broke due to the delay. One disadvantage is that the job stays just in one terminal. Could be confusing to follow with large jobs. Besides, it was the slowest one to build the website.

The decision

In the specific case of this POC, the best ones were GitHub Actions that we were already using, together with Azure DevOps. As we stopped using the process that was causing the limitation on GitHub, we continued using it.

If you want to take a look at the configuration ymls, go to my repository. Remember that it was just a POC and the ymls are very simple, but they can be a start point.

Do you disagree with any point in this text or have other experience with one of these tools? Comment here!

Which one is the best test automation tool?

Rodrigo Matola — Mon, 01 Nov 2021 18:20:50 +0000

This post was inspired by a slightly naughty response (by the context) I received in a discussion on LinkedIn, when I said that Cypress alone is not by far the best tool for API testing:

There is no better tool, but the one that meets the Project's needs." (free translation)

Yes, there is a better tool to solve your problem. And there's a better one to solve mine!

Below, I will mention 3 tools that we decided to be the best for each case, and briefly explain how the decision process was.

Puppeteer

Problem: We were building a backend tool, but it needed a browser to authenticate. Yes, there was no API for this and even CLI authentication needed to login via browser to get the credentials, which were displayed on terminal.

We were building a web application, and we needed to build a custom window.fetch object to pass as a parameter for a third-party library that we needed to use.

One week later, we still couldn't log in without using the browser. So we decided to test it from the browser anyway.

Tools: The pre-conditions was to be JavaScript tool, so Cypress, Playwright, Puppeteer and Selenium were options.

Selenium was the first to be discarded due to the need for a webdriver, unnecessary complexity and risk of incompatibilities just to log in and click a few buttons. Also, we will need to intercept requests, and Selenium does not do it natively.

Cypress and Playwright are very complete solutions for the case. Both would fit perfect, especially Cypress, as I have a certain experience on it, but could be an overkill. So, Puppeteer left.

I've never used Puppeteer before, but it's the "rawest" and lightest solution, also allows a greater flexibility. We can run it purely with Node or integrate it with any test runner we want. We chose Jest for the reason that will be explained later.

Best tool: Puppeteer

Cypress

Problem: QAs being the only ones responsible for test automation, because Selenium with Ruby and HTTParty was used for web/API automations, in a separate repository, but both front and backend developed in JavaScript.

We wanted more Devs-QAs integration.

Tools: Any JavaScript-based, like Cypress, Playwright, Nightwatch, WebdriverIO...

Cypress was chosen as the company's new stack due to its popularity growth, which allows a greater exchange of knowledge and more easy to find answers, easy installation/configuration (one command and everything is ready to go), and for being extremely friendly to all seniority. Also, an increasing number of QAs are using Cypress, which would make it easier to hire.

In addition, Cypress is an "all-in-one" tool: unit, API, component, E2E and many more built-in or via plugins functionalities.

Test code together with development code, get help from (some) devs, ease of CI, ease of mocks. As Chrome browser had more than 80% of customers usage, Cypress responded perfectly.

Best tool: Cypress.

Jest

Problem: even though Cypress was being used for front automation, we needed a tool just to do some GETs and check the response against JSON Schemas. We were building a static website and these requests were to get the content from the headless CMS.

Cypress would be a heavy tool for the above work, as we wanted to run the contract tests as the first tests in the pipeline.

Tools: Cypress, Mocha, Axios, Supertest, Chai.http, Jest.

Yes, there are plenty of other JavaScript tools to test API, but we've decided to limit the options a bit.

I really wanted to use Supertest or Chai.http in conjunction (or not) with Mocha, as this is an extremely lightweight combination.

The front was already using Axios to make requests to the CMS so a tool was already decided. Left to decide between Mocha and Jest.

React had already been decided as the stack for the frontend development, so nothing better than using Jest. Also, another Dev was already using Jest (justifying the choice said in Puppeteer section). In order not to have each project with a different tool, let's go with Jest!

And to confirm the choice, the Cypress initialization alone took longer than all the Jest contract tests...

Best tool: Jest (+ Axios and AJV)

With these three cases, we see that there is always ONE better tool to solve your problem, even if it is not THE best tool.

How do you choose your best test automation tool? Comment here!

Cyclomatic complexity: Why QAs (and Devs) should worry about it

Rodrigo Matola — Sun, 17 Oct 2021 22:43:19 +0000

This text will be the first of 3 and was inspired by a LinkedIn post, where a poll was made to know wether a developer should have a college degree. 90% voted no.

I agree with the results, but there are some contents you learn in college that can be useful at some point, and can make a difference in decision making. Three of these contents will be the subject of this series:

cyclomatic complexity;
asymptotic complexity;
binary math (focus on float numbers).

The complexities I studied in Data Structure course, and binary math in Fundamentals of Digital Computing and Introduction to Informatics (I didn't complete my degree in Computer Science).

In the texts, I'll give a straightforward and simplistic introduction to each subject, and how to use each to improve the quality of your application. At the end, I'll put links for those who want to go deeper.

What is cyclomatic complexity?

Cyclomatic complexity, or conditional complexity, is the number of independent paths an algorithm can follow. It was developed by Thomas J. McCabe in 1976, published in the article “A Complexity Measure”. (Was nostalgic to see an Fortran example in the article. It was my first programming language and I used it practically all the 12 years I spent at university).

In practical terms, it is the amount of “ifs” and “elses” that a class, method, function or equivalent has.

According to McCabe, the maximum acceptable cyclomatic complexity is 10, but searching the internet, several sources indicate the maximum between 5 and 6.

Cyclomatic complexity and tests

Cyclomatic complexity is the minimum number of tests that must be done. The greater the complexity, greater number of tests are needed, and greater the risk of bugs.

On the internet there are websites that calculate the cyclomatic complexity of a piece of code. Just copy and paste. I'll leave these 2 here:

JSHint to JavaScript
Lizard for several languages like Java, Ruby, Python and Swift

Static analysis tools such as SonarQube or SonarLint, ESLint, Rubocop also do this but you need to setup them in the project.

In practice

On a team I belong, cyclomatic complexity was used as a reference for Planning Poker scores, because unit tests was required and the devs knew that the greater the complexity, the more tests they had to do, in addition to development.

On another team, I used it as an argument for not implementing a feature. The functionality was if the person passed the number in the address search bar, the app would take that number and automatically fill in the number in field. Otherwise, the field would be empty for manual filling.

When I said that this would increase complexity, I had to explain that it wasn't the complexity of development (be harder to implement) but of testing, and that would increase the risk of bugs. I explained two or three scenarios in which there could be an error and the PO agreed not to put it.

Performance

Another point to worry about is performance. One developer I worked with always put the failure API return case in the first if, even if it happened only a few times. I always thought about how it will affect performance, even it's minimal. A time to send the request, more time to respond, more time to compare, more time to render… the person has already given up.

Searching the internet, I saw that this was a “good practice” for JavaScript, but only in some cases and just to make the code more readable. Unfortunately I didn't find anything that spoke of the performance. Not in JavaScript.

So, I found Performance Consideration For C# Conditional Statements, as the title says, is for C#. In this article, Atul Sharma do some experiments with ifs, if-else, switch-case, with match cases in the first and last statement. The experiment shows that the decision can be more than 6 times slower if true only comes in the last if-else. Do the experiment in your language too!

Good practices

I found several articles that teach how to substitute if-else and switch-case. I'll leave 3 here below, for Java, Ruby and JavaScript respectively.

If you are QA, take a look at the code of the application you are testing. Suggest changes.

If you are a Dev, try to implement best practices in your code. Use Linters and static analysis tools. And read the Clean Code book.

To know more

Appendix: Real production code with complexity 12

And another 50 warnings, measured in JSHint. Some words were removed and/or modified to not identify the product and translated to English.

function checkNavigation() {
    _event(
      'new_services',
      'button',
      isEmpty(title) === false ? title.toLowerCase() : subtitle.toLowerCase(),
    )
    if (typePos) {
      if (!hasTicket) {
        if (payment) {
          Alert.alert('OK', "You don't have any open invoice!")
        } else if (alertOpen) {
          Alert.alert('OK', "You don't have any open invoice!")
        } else {
          if (isEmpty(propsNavigate) === true) {
            navigation.navigate(navigateTo)
          } else {
            console.log('propsNavigate--> 1', propsNavigate)
            navigation.navigate(navigateTo, propsNavigate)
          }
        }
      } else if (alertOpen) {
        openAlert()
      } else if (hasTicket) {
        if (isEmpty(propsNavigate) === true) {
          navigation.navigate(navigateTo)
        } else {
          console.log('propsNavigate--> 2', propsNavigate)
          navigation.navigate(navigateTo, propsNavigate)
        }
      }
    } else if (type3) {
      if (Store.getState().extractPreReducer.balance === '0') {
        Alert.alert(
          "You don't have any open debts!",
          [
            {
              text: 'Close',
            },
            {
              text: 'Further informations',
              onPress: () => {
                navigation.navigate('HowWorkScreen')
              },
            },
          ],
        )
      }
    } else {
      navigation.navigate(navigateTo)
    }
  }

Do you know testing? Maybe it’s not enough...

Rodrigo Matola — Fri, 09 Apr 2021 16:15:42 +0000

Originally written at my LinkedIn on September 6, 2016. I had just finished my first job with software testing.

I was thinking of writing this article when I read this other here Do software testers need to learn coding?. They complement each other.

I hear sometimes at work: this is not my problem, I don’t need to know that, it’s not my task and I don’t care. Yes, you will not meddle in the task / work of others, but a project where several areas are involved, we have to get a sense of the role and what makes each project area. This sense needs to be deeper if these areas work directly with each other.

In my case for example, we test the results software. Keeping simple, there are two different companies, one for getting and one for dissemination of results. So I have to deal with two different ways of working (very different).

At the beginning I was told to just do my job (manual testing). Anything needed to be made in the system, was necessary to call someone or open a ticket. This sometimes caused a break in testing, because we depended on others to continue.

Over time, I learned the processes and procedures that helped me to improve my testing and save time. Many defects opened to a system was in fact from another, which have the previous processing. Unfortunately this was not taught and had to run back on its own, on trial and error, asking people who knew more (or should), sometimes with vague answers.

Black box testing is sometimes taken very seriously. You test only what you see, no idea what's behind. By analogy, if your computer crashes is not better you understand how your work tool works? You do not need to know how to fix, but this knowledge will help you to argue with the technician who may want to fool you or make a kludge.

Anybody know anything about internal combustion engines? Of couse! Anybody know how to fix an internal combustion engine? No... (The Big Bang Theory)

We are not only in the testing department, or development, or software sales. We are in the software delivery business, and to deliver the best quality software, different areas and companies should work integrated as a single team.

"Teamwork is not when everyone works together, but when we trust each other." (Unknown)

DEV Community: Rodrigo Matola

Stop using Appium (if you have access to the source code)

Context

The idea

Appium X Native Automation

Advantages of Appium

Disadvantages of Appium

Disadvantages in my context

Native automation advantages

Disadvantages native automation

Who writes the BDD?

BDD, Cucumber, Gherkin

BDD

Cucumber

Gherkin

You don't do BDD

References

Number of Devs X Number of QAs: Why do I think it's an ineffective measure?

Task team

Correct metric: features / QA

Find your number

NEVER trust in unattended data

Contextualizing

Android or iOS?

Time of section

Average alone means nothing

Use other central tendency measurements

With that alone, can I make an assertive analysis?

Conclusion

How I automate tests (or try to) inside Sprint

Company

Team

Imagination

My flow

Implement the scenarios

Don't forget the exploratory manual test

Further readings:

Automation metrics: some problems

The 3 numbers

Only numbers don't mean anything

It will never be 100% automated

To achieve the goals, we will hire "automators"

Conclusion

Ruby: "the best" language for general automation

Why Ruby?

Where is Ruby used?

My experience with Ruby

Continuous compliance

GitHub Automations

CLI Test Automation

Conclusion

Where to learn Ruby?

Ready, Accepted, Done! Acceptance criteria and definitions of ready and done.

Acceptance Criteria (AC)

Definitions

Example

Definition of Ready (DOR)

Definition

Examples

Explaining the examples:

Definition of Done (DOD)

Definition

Examples

Explaining the examples:

Azure, CircleCI, GitHub, Gitlab: a POC

CircleCI

CircleCI Pros*

Cons

My opinion

Azure DevOps

Azure DevOps Pros

Cons

Gitlab CI

Gitlab CI Pros

Cons

GitHub Actions (GHA)

Summary

The decision

Which one is the best test automation tool?

Puppeteer