Try AREX for Shadow Testing with Zero Test Case Writing

As business systems evolve and become increasingly complex, quality engineers are constantly on the run. Traditional functional testing or automated testing practices no longer provide sufficient confidence in quality assurance, shadowed by the pain points of quality activities. These can be summarized as follows:

1. High script maintenance cost: The absence of comprehensive interface contract documentation makes the implementation of interface automation challenging, leading to high costs in writing and maintaining test cases due to iterative changes.
2. Tight project timelines: Insufficient testing regression.
3. Inadequate personnel experience & difficulty in test data creation: Script parameters fail to simulate real-world scenarios, making it harder to reproduce bugs.
4. Complex traffic: High cost in constructing requests, leading to incomplete coverage of test scenarios.
5. Business complexity: Difficulty in organizing business scenarios, with unrealistic traffic distribution.

Shadow Testing is a technique that aims to replicate the production environment (referred to as V-Current) in order to simulate real user traffic. This is achieved by creating a shadow version of the production environment. The purpose of this approach is to test new features in the V-Next environment. Once the testing phase is completed, the responses from both environments are compared to mitigate any potential risks before deploying the new feature to the production environment.
Shadow Testing has gained significant popularity within the testing community, mainly due to its effectiveness in conducting regression testing for large-scale services. It offers extensive coverage while keeping maintenance costs to a minimum.

Although similar in its objective, the AREX implementation of regression testing differs slightly from shadow testing.

Capture production traffic

A crucial part of shadow testing is testing with production data.

Java Agent is a powerful tool based on the JVM (Java Virtual Machine) that allows for the modification and enhancement of Java application bytecode at runtime, without the need to alter the application's source code. This capability makes Java Agent an ideal choice for implementing features such as monitoring, performance analysis, and traffic recording. By leveraging the Java Instrumentation API, agents can transform class files loaded by the JVM, enabling a wide range of dynamic analysis and manipulation tasks.

Understanding Traffic Capturing with Java Agent

Traffic capturing refers to the process of capturing the input and output data of an application during its execution. This includes network requests, responses, and potentially database queries, among other types of interactions. It is invaluable for debugging, performance optimization, security testing, and more, offering insights into the application's behavior under various conditions.

Bytecode Instrumentation

AREX Java Agent implements traffic recording primarily through bytecode instrumentation. Libraries such as ASM or Javassist are used to manipulate specific classes and methods at the bytecode level. This process, known as instrumentation, allows the agent to insert additional functionality for recording traffic without altering the existing business logic.

Data Capture

During instrumentation, Java Agent can inject extra code at the entry and exit points of methods to capture input parameters and return values. For network requests and responses, enhancing classes and methods related to networking libraries enables the capture of this data. This approach ensures that the traffic recording is comprehensive and includes all relevant data flows within the application.

Serialization and Storage

Once captured, the data needs to be serialized and stored for later access and analysis. Java Agent can be configured to use various serialization formats, such as JSON or XML, and storage mechanisms, including local files or remote databases. This flexibility allows developers to tailor the traffic recording to their specific needs, facilitating easier debugging and performance tuning.

Running tests and saving outputs

Once we have the input data, the next step is to build and run the test.

By leveraging the Java Agent to capture input parameters, we can accurately simulate requests to the application under test. Typically, the application being tested is also equipped with an agent. This setup allows for a nuanced approach to traffic replay where the primary interface calls are executed genuinely, while the sub-calls to third-party dependencies are not.

During replay, if a sub-call's input parameters match those recorded during the traffic recording phase, the recorded data is returned directly, effectively mocking the sub-call. This method of replaying traffic by using Java Agent to mock sub-calls based on identical input parameters ensures a high fidelity simulation of the production environment.

Advantages of This Approach

The primary benefit of this strategy is the ability to recreate the production environment's conditions in the test environment without making actual calls to third-party services. This not only helps in isolating the application under test but also significantly reduces the complexity and cost associated with setting up external dependencies for testing purposes. Moreover, it allows developers and testers to focus on the application's behavior in conditions that closely mirror those found in production, facilitating more accurate performance evaluations and debugging processes.

Comparing The Differences

Now that we have the test result and the result of the production version. We need to find the differences.

AREX not only compares the parameters and responses of the main-interface requests but also compares the request marameters of external dependencies.

A practical Example: Processing a Product Purchase Request

Consider a RESTful service with an endpoint /purchaseProduct that handles product purchase requests. When this endpoint is hit with an HTTP request, it triggers a method processPurchase in the service to handle the purchase logic, including database interactions to check and update product stock.

It involves both an entry point call and sub-calls to a database, providing a comprehensive example of traffic recording and replay using the AREX Java Agent.

/**
 * Processes a purchase request for a given product.
 * It involves an HTTP request as the entry point and a database operation as the sub-call.
 *
 * @param productId The ID of the product being purchased.
 * @param quantity The quantity of the product being bought.
 * @return A response indicating the success or failure of the purchase.
 */
@PostMapping("/purchaseProduct")
public ResponseEntity<?> processPurchase(@RequestParam Long productId, @RequestParam int quantity) {
    // Entrance Invocation: HTTP request to purchase a product
    
    // SubInvocation 1: Check product availability in the database
    int availableStock = productRepository.getStockForProduct(productId);
    
    if (availableStock >= quantity) {
        // SubInvocation 2: Update product stock in the database
        boolean isUpdated = productRepository.updateProductStock(productId, availableStock - quantity);
        if (isUpdated) {
            return ResponseEntity.ok("Purchase successful.");
        }
    }
    
    return ResponseEntity.status(HttpStatus.BAD_REQUEST).body("Insufficient stock.");
}

Step 1: Traffic Recording

Recording the Request: The Java Agent begins by recording the incoming HTTP request for a product purchase, capturing details such as the request method, URL, parameters (productId, quantity), and any headers or cookies if applicable.

Recording Database Operations: The agent then records the database queries made by the processPurchase method. This includes checking the product's available stock and attempting to update the stock if sufficient. Each database operation's request and result are captured.

Step 2: Traffic Replay

Replaying the Request: During the replay phase, the HTTP request to /purchaseProduct is made genuinely, reflecting a real-world interaction as would occur in production. This ensures that the test conditions closely mimic actual user behavior.

Mocking Database Operations: For database interactions, the agent checks if the product ID and quantity match the recorded data. If they do, it mocks the database responses based on the recorded outcomes, thereby simulating the database's behavior for both checking and updating stock without real database access.

Conclusion

To sum up, there are a lot of benefits to using AREX:

• Zero impact on the production environment.
• No need to generate and maintain test cases and test data.
• Test real-life scenarios with real-life data.
• Great test coverage from using production data.