Expect to Inspect – Performing Code Inspections on Your Automation [Testμ 2023]

In today’s digital landscape, it’s crucial to view automation as software development. Despite using drag-and-drop or record-and-playback interfaces, underlying code structures drive these processes. To avoid pitfalls of unsustainability and premature project failure, it’s important to approach automation as a software development endeavor. As software development benefits from code inspections or reviews, where different team members identify issues and risks in newly written or modified code, automation should undergo the same process.

In this session, Paul Grizzaffi deep dives into the significance of code inspections for automation software, the distinctions between automation and product software inspections, and real-life issues through these examinations.

About the Speaker

Paul Grizzaffi, a Senior Automation Architect at Vaco, loves using technology to help testing and quality assurance teams. He helps them with things like automation projects and making their work better. An accomplished keynote speaker and writer, Paul has spoken at both local and national conferences and meetings. He is an advisor to Software Test Professionals and STPCon and a member of the Industry Advisory Board of the Advanced Research Center for Software Testing and Quality Assurance (STQA) at UT Dallas, where he is a frequent guest lecturer. Paul shares his thoughts and experiences with other testers through his blogs. You can check out his blogs at https://responsibleautomation.wordpress.com/.

If you couldn’t catch all the sessions live, don’t worry! You can access the recordings at your convenience by visiting the LambdaTest YouTube Channel.

Session Overview

After introducing himself, Paul Grizzafi started the session by mentioning automation is software development. While comparing automation and software development, he highlighted the similarities between the two- having a plan, doing the development work, and testing the created solution. He talked about the significance of testing automation to ensure it performs as expected, as automation involves trusting computer-based processes. Paul further showcased the importance of treating automation seriously and avoiding potential problems by not considering it as a software development endeavor.

He discussed the different phases of software development, including development, testing, and supporting the software after it’s created. He emphasized the need to store the code somewhere, deploy it to the intended environment, manage different versions, and inspect the code for its fitness and purpose. This inspection and code review aspect is what he focused on during the session. He highlighted that the session would focus on inspecting automation code, just as one would inspect software code in traditional development processes.

What is Code Inspection?

As the session progressed, Paul talked about Code Inspection or Code Review. Basically, code inspection is defined as a systematic evaluation of code, similar to how one seeks help when debugging a problem. Just as an editor reviews a book for errors, code inspection involves having a second set of eyes scan the code to identify existing and possible issues.

It serves as a mechanism to identify problems and ensure code quality. Code inspections can vary in formality, ranging from detailed line-by-line reviews in a pull request to informal discussions where changes are explained during a screen share. According to Paul, the key is to choose an appropriate approach that matches the nature and risk level of the code changes being made.

Why Should You Perform Code Inspection?

Although Paul specifies that there could be various other reasons for conducting code inspection, he discussed 5 top purposes for performing it:

Error Detection: Code inspections are primarily conducted to identify errors or issues in the code that might have been missed during initial testing or development. Another engineer can spot the errors that the original developer might have overlooked, such as off-by-one errors or other coding mistakes.

Maintenance and Maintainability: Code that seems straightforward to one developer might not be as clear to others. Inspections help identify areas of the code that could be improved for better maintainability. For instance, a reviewer might point out the need to refactor certain sections for improved future maintainability or to handle potential changes more efficiently.

Risk Assessment: Code inspections also involve checking for any potential risks associated with the code. This also includes whether the code will perform well as the user base grows or if any potential performance bottlenecks, bugs, or crashes could occur under different scenarios.

Classification and Tagging: Inspections can help classify code and add appropriate tags. This is useful for categorizing code scripts for testing suites like smoke or regression testing. Tagging enables efficient test execution when specific system parts are updated or modified.

Cross-Training: While not the primary goal, involving different team members in code inspections, especially new joiners or those unfamiliar with the codebase, can offer some benefits. It can provide a different perspective and help new team members understand the codebase better. However, Paul also noted that code inspections are not a proper way to learn the codebase.

How to Approach Code Inspection?

Paul shared his insights on how to approach code inspection. He believes that it should be an inclusive and collaborative process. He suggested involving team members and even the customers during code inspection to make well-informed decisions and ensure the quality and effectiveness of the codebase. Let’s learn his way of doing it:

Inspect All Code: Paul emphasized the importance of inspecting all lines of code, though the level of formality may vary based on the perceived risk. While some minor changes might not require an elaborate review, it’s generally advisable to involve a second set of eyes for all code changes, regardless of size.

Stewardship of Quality: He introduced the concept of ‘stewardship’ of quality, indicating that the testing team’s role is not solely to be responsible for quality but to act as stewards who ensure quality is upheld across the entire team. This involves making decisions that align with maintaining an appropriate risk profile for the team.

Everyone is a Reviewer: Paul stressed that anyone within the team can review the code; it should not be restricted to just the original tester. This applies not only to product or application code but also to test automation code. He further said that their opinion should be valued by everyone reviewing the code.

Involving Users: While considering test automation, involving users is essential. Test automation teams often work closely with testers directly interacting with the automation. These testers are the users of the automation tools. Their input is crucial in evaluating the effectiveness and usability of the automation solutions developed by the automation team.

Why Should Testers Participate?

Paul has noticed many test engineers don’t want to be a part of the code review process since they are not programmers. Paul encourages testers to participate in code reviews for test automation, even if they aren’t programmers. He highlights the value of functional validation by having testers run the automation to observe if it behaves as expected, comparing it with manual actions.

While testers don’t need extensive programming knowledge, a basic grasp aids in effective validation. Testers’ involvement ensures the automation meets their needs, making small changes if required, and boosts efficiency by reducing manual retesting. Paul reminded everyone that reviewing automation differs from app code, focusing on its reliability for testing tasks. He wants testers to engage in the process to ensure the automation is understandable and dependable for their testing work.

Why Should Programmers Participate?

Coming to programmers, Paul suggested that programmers should also participate in code reviews for test automation. Having fellow developers or automation experts with programming skills review the code is valuable. They can help ensure correctness and adherence to coding standards and catch potential performance issues or pitfalls. Having a fresh perspective can reveal problems not caught through regular testing methods.

Paul emphasized that involving programmers in code inspection can prevent issues like unexpected crashes due to simple mistakes, like adding punctuation. Their expertise can contribute to reliable and efficient automation that aligns with coding standards and performance considerations.

What to Look for During Code Inspection?

Paul gave a clear overview of what to look at during code inspection. He highlighted two major categories: Static Analysis and Dynamic Analysis.

• Static Analysis: Paul explained that in static analysis, you look for errors or potential errors and adhere to the coding standards and the presence of any warning.

• Dynamic Analysis: Paul compared it to testing but with a narrower focus on code quality. Testers should run the automation themselves and observe if it behaves as expected. He said that timing the automation can help identify performance issues.

Paul also emphasized the importance of using tools while code inspection. During the session, he also shared a list of tools to help enforce the coding standards, detect errors, and provide feedback on code quality.

Why should I Do a Code Inspection?

While Paul has been a true advocate for code inspection, he has often been asked certain questions like I have unit tests to check everything for me, so why should I perform code inspection? or What is the ultimate goal of code inspection?

Answering the first objection, Paul acknowledged the presence of unit tests but emphasized that those unit tests are also code and can have bugs. He suggested that while unit tests are valuable, reviewing them is still important, especially for critical or high-stakes systems.

Ultimately, Paul highlights that the goal of code inspection is to identify potential and actual issues, including bugs that can be easily overlooked or dismissed as careless mistakes. The emphasis is on improving the quality and reliability of the automation code.

What is the Importance of Pair Programming in Code Inspection?
Paul explained Pair Programming as involving two individuals: one who works on the code and the other who works as a reviewer and provider of inputs. Talking about its significance, he said pair programming could provide a collaborative approach to detecting bugs and anomalies in real time. However, whether an additional set of eyes is needed through code inspection depends on factors such as the level of risk and the criticality of the developed system.

For less critical systems, pair programming might be sufficient to catch issues. However, involving another person in a code review can be valuable for more critical systems, as it offers an additional layer of scrutiny.

What Kind of Bugs can be Expected While Code Inspection?

After discussing the why and how of code inspection, Paul talked about the bugs that could be spotted during code inspection. These are the three bugs he mainly talked about:

Boolean Bug Assignment: He highlighted the “Boolean assignment bug,” where an assignment is used within a conditional statement, leading to unexpected behavior due to how the algorithm works. This bug is exemplified with a code snippet in languages like C++ and C that allow such assignments within conditionals.

Off by One Error: When loops or conditions are not properly adjusted, causing array index out-of-bounds exceptions or other issues, off by one error occurs. Static analysis tools can help catch some of these bugs, but the speaker emphasizes that such tools might not be as effective in some cases.

Log and Error: Paul talked about how logs and errors are crucial for understanding what’s happening in automation runs. While we can debug and find issues on our personal computers, we need logs to know what’s going on when tests run without us. The first alert catches problems but doesn’t tell us much, and the second one gives more info but can still be confusing. If we’re not good with code, we might struggle. It’s important to review code, especially when things are complicated carefully. Also, tidying up repeated code is a good idea, even though tools might not help with naming things better.

Can AI do Code Inspection Accurately?

Paul discussed that AI can take over code creation and code checking. To clear this misconception, Paul used a practical example of presenting a code snippet where there might be a bug in the form of a mismatch between the intended number of iterations and the actual number of iterations. He then demonstrates how they interact with ChatGPT to identify the bug.

This example confirmed the inconsistencies in the AI’s responses. In one instance, AI claimed there was no bug, while in another, it identified the bug and even provided a potential fix.

Paul emphasizes that AI tools can be useful assistants but can not replace human minds. While AI can be valuable for various tasks, it’s important to maintain human oversight and critical evaluation, as AI may not always provide accurate or consistent results, especially for intricate coding scenarios.

How to Plan Code Inspection Effectively?

Many of us operate within agile methodologies or ticket-based workflows, requiring careful task allocation. Here’s how Paul broke down the process:

Inspection Tasks: Firstly, tasks should be designated for the inspection. This involves analyzing the code thoroughly to identify any issues, bugs, or areas for improvement.

Rework Tasks: It’s important to anticipate that some parts of the code might need rework based on the insights gained during the inspection and review. Therefore, assigning tasks for code rework ensures that identified issues are addressed promptly.

Retest Tasks: After the rework, the code needs to be retested to verify that the changes have been made correctly and that no new issues have arisen. This step ensures that the code remains functional and stable even after modifications.

Rereview Tasks: In some cases, particularly when dealing with sensitive parts of the code or significant changes, involving a fresh set of eyes might be beneficial. Assigning a task for a second review can help ensure the code changes are accurate, maintainable, and aligned with the intended goals.

In the end, Paul concluded by asking everyone to remain kind during the code review process, highlighting that the goal is improving the code and enhancing the overall user experience rather than getting caught up in minor details like indentation.

He encouraged the audience to consider following established coding standards and rules for their programming language, which can often be found through a quick online search. He further reiterated the importance of treating automation like any other software development process and making code inspection a positive and value-driven experience.

Time for Some Q&A

Q. How can the entry and exit criteria be best defined in a code inspection?

Paul: “Entry Criteria” usually means the automated steps I take before a test, and “exit criteria” involves wrapping up setup and related actions. I think the best approach depends on the situation, like parameterizing steps or using separate methods/functions. Be kind during code reviews and improve the code’s quality.

Q. What is the best way to inform a team member that the code is not up to the mark?

Paul: To inform a team member that their code needs improvement, you can use analysis tools whenever possible to identify issues. Let these tools point out problems and act as the “bad guy.” If the tools can’t address all aspects, approach it empathetically. Put yourself in their shoes and offer feedback like, “I found some areas where we can make maintenance easier” or “I think there might be a bug. Could you explain?” Humanizing the feedback helps maintain a positive atmosphere while addressing concerns effectively.

Can you suggest any platforms that can help practice to improve code quality?

Paul: For improving code quality, use static analysis tools to highlight issues and guide improvements. If you’re seeking training or educational platforms, trustworthy options like Pluralsight, Udemy, and Test Automation University offer valuable resources to enhance your coding skills. Test Automation University is free and focuses on test automation topics.

How do you balance code review and business alignment during code inspection for automation, ensuring scripts align with technical standards and strategic goals?

Paul: Balancing code review with business alignment in automation code inspection involves considering technical standards and strategic goals. While ensuring code correctness is crucial, it’s essential to prioritize the overall behavior of the code. Ultimately, whether to release software or address potential issues lies with the decision-makers who rely on information provided by testers and the team. Human skills and expertise are at the heart of this process, aligning code quality with business objectives and achieving the organization’s goals.

Have you got more questions? Drop them on the LambdaTest Community.