Discussion: Challenges that we face as dev test engineers!?

I have worked on projects where the customer literally requests: We need to automate the testing for this project!

Basically, this makes me ask:

What do you exactly need me to do?

• What are the goals?
• What are the priorities?
• How do you need me to integrate with the SLC!?

What other challenges do you face as test engineers?

Comments

[Ken Simeon]

Great topic, Pablo!

Here are many of the challenges I've faced over the years of building automated testing.

1) Creating & supporting your test environment(s).

• When it comes to running test automation that are integrated with Ci/CD system ecosystems, it falls to the quality engineer (aka dev test engineer) to be the keeper of the gold key. They must have the system administrative knowledge or gain the knowledge to maintain the automated test infrastructure. Gone are the days that automated tests only run on the solo machine under the hands of the quality engineer. Today, test code is production code. It must be developed and treated the same.

2) To go along with your 3rd question, Will a failure of the automated tests ever stop a release?

• This question and your's comes down to, will the automated test ever be trusted to truly measure the quality of the software that has potential to be release to the public.

3) How do you get others to contribute to the automation test framework and coverage?

• Much of the expansion of the test framework and overall coverage has to come for the development leaders pushing or mandating the developers to expand their code bases' test coverage. Yes a test engineer or group can expand the coverage of a platform, but it will be slower then if the whole development staff was responsible for adding to the test coverage. Then its on the test engineer to set the ground rules for proper test writing and then be the test code police.

4) How do you make the reporting of the success/failure of the automated tests publicly consumable?

• The quickest way to make the success/failure results known to all in the development groups is to tie the results into your CI/CD system. Ideally your DevOps team or release manager would be the eagle eyes of the test results to sound any alarms. But that typically isn't enough. Most test engineers need to have a way of reporting the growth or decline of a platforms software quality over time. Tracking the trends is just as important as the day-to-day results.