Software Verification and Validation in The Process

• Introduction

• Testing Types

  • Validation
    • Functional
      • Unit Testing
      • Integration Testing
      • System Testing
      • Regression Testing
      • Object-Oriented Testing
      • Sanity Testing
      • Smoke Testing
      • GUI Testing
    • Non-Functional
      • Usability Testing
      • Security Testing
      • Performance Testing
        • Scalability
        • Stress
        • Load Testing
        • Stability Testing
      • Installation Testing
  • Verification
    • Acceptance Testing
    • Alpha Testing
    • End-to-end Tests
    • User Testing

• Software Testing Activities

  • Requirement analysis.
  • Test planning.
  • Test case identification, design and development.
  • Coverage
    • Human-Based
    • Criteria-Based
  • Test environment setup.
  • Test execution.
  • Test cycle closure.

Introduction

TODO: Complete this section

Testing Types

TODO: Complete this section

Testing can be categorized (in a very high-level) based on the knowledge of the system under test as either black, white, or grey box testing.

White box testing (also known as clear, glass box or structural testing) is a testing technique which evaluates the code and the internal structure of a program.

Black box testing (also known as functional testing) is a testing technique which ignores the internal parts and focus on the output is as expected or not.

Grey box testing is the combination of both White Box and Black Box Testing.

Source from Javatpoint

Verification: “Are we building the product right?”
Validation: “Are we building the right product?”

Validation

Validation is intended to ensure a product, service, or system (or portion thereof, or set thereof) results in a product, service, or system (or portion thereof, or set thereof) that meets the operational needs of the user. It consists of two types, functional or non-functional testing.

Functional Testing

Functional testing is a type of testing which verifies that each function of the software application operates in conformance with the requirement specification. This testing mainly involves black box testing and it is not concerned about the source code of the application.

Non-Functional Testing

Verification

Verification is intended to check that a product, service, or system meets a set of design specifications.

Software Testing Activities

After we knew most testing types, now we deep in software testing activities. They are the various tasks and processes involved in ensuring that a software product meets specified requirements and works as expected. These activities include:

Requirement Analysis

Requirement Analysis is a crucial activity that involves understanding and analyzing the requirements of the software system to be tested. It helps in identifying the scope of testing, defining test objectives, and ensuring that the testing process aligns with the project's goals. Here are some details about the Requirement Analysis activity and best practices:

• Understanding Requirements:

The first step in Requirement Analysis is to thoroughly understand the software requirements. This involves reviewing the functional and non-functional requirements, user stories, use cases, and any other relevant documentation.

• Clarifying Ambiguities:

During Requirement Analysis, it is essential to identify and clarify any ambiguities or inconsistencies in the requirements. This can be done by discussing with stakeholders, developers, and business analysts to ensure a clear understanding of the expected behavior of the software.

• Identifying Testable Requirements:

Testable requirements are those that can be validated through testing. It is important to identify and prioritize the testable requirements to ensure comprehensive test coverage. This can be done by mapping requirements to test cases or test scenarios.

• Defining Test Objectives:

Based on the requirements, the testing team should define clear and measurable test objectives. These objectives should align with the project goals and help in evaluating the quality of the software system.

• Identifying Risks:

Requirement Analysis should also involve identifying potential risks and their impact on the testing process. This includes understanding dependencies, constraints, and potential challenges that may affect the testing effort.

• Collaboration and Communication:

Effective collaboration and communication with stakeholders, developers, and business analysts are crucial during Requirement Analysis. Regular meetings, discussions, and documentation should be used to ensure a shared understanding of the requirements.

• Documentation:

It is important to document the analyzed requirements, including any assumptions, constraints, or decisions made during the process. This documentation serves as a reference for the testing team and helps in maintaining traceability between requirements and test cases.

• Continuous Requirement Review:

Requirement Analysis is an iterative process, and it is important to continuously review and update the requirements as the project progresses. This helps in accommodating changes, addressing new insights, and ensuring that the testing process remains aligned with the evolving requirements.

Test Planning

The goal of this activity is to define the overall approach and strategy for testing a software system. It involves creating a comprehensive plan that outlines the scope, objectives, resources, and timelines for the testing process.

During the Test Planning activity, you can perform several tasks, including:

• Defining the test objectives:

Clearly state what you aim to achieve through testing, such as identifying defects, ensuring system functionality, or validating specific requirements.

• Identifying the scope:

Determine the boundaries of the testing effort, including which features, modules, or components will be tested and which will be excluded.

• Creating a test strategy:

Develop a high-level approach to testing, considering factors like test levels (unit, integration, system, etc.), test types (functional, performance, security, etc.), and the use of manual and automated testing techniques.

• Estimating resources:

Determine the number of testers, test environments, tools, and equipment required for the testing process. This includes considering the availability of hardware, software, and personnel.

• Defining test deliverables:

Specify the documents and artifacts that will be produced during testing, such as test plans, test cases, test scripts, and test reports.

• Establishing test schedules:

Create a timeline that outlines the sequence of testing activities, milestones, and deadlines. This helps in coordinating testing efforts with other project activities.

• Risk assessment and mitigation:

Identify potential risks and challenges that may impact the testing process. Develop strategies to mitigate these risks and ensure smooth testing operations.

Best practices for test planning include:

• Involving stakeholders: Collaborate with project managers, developers, business analysts, and other relevant stakeholders to gather input and ensure alignment with project goals.
• Prioritizing test objectives: Clearly define the most critical aspects of the software system to focus testing efforts on areas that are most likely to have a significant impact on the system's quality.
• Considering test coverage: Aim for comprehensive coverage by selecting appropriate test techniques and ensuring that all critical functionalities, use cases, and scenarios are addressed.
• Adapting to project constraints: Take into account project timelines, budget limitations, and resource availability while planning the testing activities.
• Reviewing and refining the plan: Regularly review and update the test plan as the project progresses, incorporating feedback and lessons learned from previous testing phases.

To create a good test plan, consider the following steps:

• Understand the project requirements and objectives thoroughly.
• Identify the key stakeholders and involve them in the planning process.
• Define clear and measurable test objectives.
• Determine the scope and boundaries of the testing effort.
• Select appropriate test techniques and strategies based on the project's needs.
• Estimate the required resources and create a realistic schedule.
• Identify and mitigate potential risks.
• Document the plan in a clear and concise manner, ensuring it is easily understandable by all team members.

Test Case Identification, Design, and Development

Here, the testing team designs the test cases based on the software requirements. The test cases specify the conditions under which a tester will determine whether an application, software system, or one of its features is working as it was originally established for it to do.

Test Case Design and Development activity involves creating detailed test cases that will be used to verify the functionality of the software being tested.

The goal of this activity is to ensure that all aspects of the software are thoroughly tested and that any defects or issues are identified and addressed.

During this activity, you can perform the following tasks:

• Test Case Identification:

Identify the specific test scenarios and conditions that need to be covered by the test cases.

• Test Case Design:

Create and document the actual test cases, including the steps to be executed, the expected results, and any necessary test data or preconditions.

• Test Case Development:

Start developing the test code if it is designed to be code. You can start writing unit and integration tests here.

To ensure a well-structured and effective test case identification, design, and documentation, consider the following best practices:

• Test Coverage:

Ensure that the test cases cover all the important functionalities and features of the software. This can be achieved by using techniques like equivalence partitioning, boundary value analysis, and decision table testing. Also, you can design test cases for best coverage based on the return values and exceptions that may be thrown that is in the docstrings of functions and classes.

Types include:

1. Human-based coverage:

This type of test coverage focuses on the expertise and intuition of human testers. It involves the testers' ability to identify potential risks, vulnerabilities, and areas of the system that require thorough testing. Human-based coverage relies on the knowledge and experience of the testers to ensure that critical functionalities and scenarios are adequately tested. Testers may use their understanding of the system, user behavior, and potential edge cases to design test cases and execute them.

1. Criteria-based coverage:

This type of test coverage is based on predefined criteria or specifications. It involves systematically testing the system against a set of predetermined conditions or requirements. Criteria-based coverage ensures that all specified functionalities, features, and scenarios are tested. This approach typically involves creating test cases based on requirements, specifications, or user stories. Examples of criteria-based coverage techniques include statement coverage, branch coverage, path coverage, and decision coverage.

• Clarity and Simplicity:

Write test cases in a clear and concise manner, using simple language and avoiding ambiguity. This helps testers understand and execute the test cases accurately.

• Reusability:

Design test cases that can be reused across different test cycles or for regression testing. This saves time and effort in the long run.

• Traceability:

Establish traceability between test cases and requirements, ensuring that each requirement is covered by at least one test case. This helps in verifying the completeness of the testing process.

• Prioritization:

Prioritize test cases based on risk, importance, or criticality. This ensures that the most critical functionalities are tested first.

• Maintainability:

Regularly review and update test cases to keep them up-to-date with any changes in the software or requirements.

Model-Driven Design Testing

Model-driven testing is a methodology that uses models to represent the functionality of the system under test. These models are used to guide the design of the test cases.

Test Environment Setup

A test environment is created according to the requirements of the software. This might include setting up of the server, client machines, operating system, hardware, software and network connectivity.

Test Execution

The test cases are executed in the testing environment, whether it is functional or nonfunctional During this stage, the testers execute tests manually or developers and testers start running code that is built in the design and development phase.

Data-Driven Testing

Data-driven testing is a methodology that separates the test script logic and the test data. It allows you to test how the application handles various inputs effectively.

Test Automation

Making tests as code and integrating tests in CI/CD pipelines (That we will discuss in the next article) will automate the testing execution activity.

So, for examples when finishing the development phase and before integrating the code to the system,

you can run unit and integration tests.

After that, you can run other functional testing and system tests.

After building the main code, you may run smoke , nonfunctional, and regression testing.

Test Cycle Closure

In this final stage, the validationthe testing team meets to evaluate the cycle completion criteria based on Test coverage, Quality, Cost, Time, Critical Business Objectives, and Software. They discuss what went well, which areas need to be improved and what can be learned from the project. Also, you can create test reports for developers to retest and evaluate next tests.

Conclusion

Software testing is a vast field with many different aspects. This guide provides a comprehensive overview of the various activities, models, types, and tools involved in software testing. Whether you're a seasoned tester or just starting out in the field, understanding these concepts can help you to design and execute effective tests, ensuring that the software you're testing is of the highest possible quality.

References:

• Javatpoint
• Wikipedia

---

Real-World Skills and Diving Deeper

• Quality Engineer Roadmap

---

Jobs:

1. Software Tester
2. Software Developer
3. Quality Assurance Engineer
4. Peneteration Tester

---

What's Next?

Next activity is the Evolution and Maintainance, where we start making the the process of evolving and automating the software process.

---

TODO:

• Give details in designing and how to design the flow and make test cases and better coverage with the less test cases possible
• Give examples with python
• Give details in types