How Can Unity Teams Analyze Performance Across Multiple Test Devices at Once?

Summary

As game teams increasingly run automated performance tests across multiple physical devices at the same time, the amount of test data can quickly grow from a few reports to dozens or even hundreds.

Reviewing these reports one by one is time-consuming and makes it difficult to quickly determine which devices, test cases, or performance metrics require attention.

GOT Online now provides Batch Test Analysis, which automatically analyzes multiple test reports and generates an Excel file containing an overall test summary, optimization tasks, affected device models, test failures, and performance data.

This allows development teams to move from reviewing individual reports to identifying common performance problems across an entire batch of automated tests.

Introduction

A typical automated performance testing workflow may look like this:

Build the game → Install it on multiple physical devices → Run automated tests → Collect performance data → Upload the data to GameOptim → Analyze the results

Running tests in parallel across many devices can significantly reduce testing time and increase scenario coverage.

However, this approach creates a new problem: how do you efficiently analyze dozens or hundreds of test results?

There are usually two questions that development teams need to answer:

1. Which devices and test cases have performance problems?
2. Did any functional problems occur during testing, such as crashes or blocked gameplay?

To address these problems, GOT Online introduces Batch Test Analysis.

How Can Developers Analyze Multiple GOT Online Reports at Once?

In an Overview mode report, developers can select Batch Test Analysis, choose the test data to analyze, and click Start Analysis.

The system then automatically generates an Excel workbook containing five sheets:

1. Test Overview
2. Optimization Queue Details
3. Devices with Significant Performance Issues
4. Test Failure Details
5. Performance Reports

Each sheet provides a different perspective on the batch test results.

What Can the Test Overview Tell You?

The Test Overview sheet provides a high-level summary of the overall test results.

For example, in the following test, the compatibility pass rate is 40%. Among the 10 analyzed tests:

• 4 passed the performance criteria
• 5 failed the performance criteria
• 1 exited unexpectedly due to a crash

Instead of opening each report individually, developers can use these summary results to quickly understand how the project performs across the tested devices.

This is especially useful when a test batch contains a large number of devices.

How Can Batch Analysis Identify Common Performance Problems?

The Optimization Queue Details sheet summarizes performance problems that appear across the analyzed test data.

Rather than simply reporting that a test failed, the analysis provides more detail about the performance bottlenecks, including the affected modules and devices.

This gives the development team a clearer picture of which performance problems occur repeatedly across the test batch.

It also makes optimization prioritization more meaningful.

For example, a performance issue that appears across many devices may deserve higher priority than an issue that occurs in only one specific test case.

Can Batch Analysis Identify the Specific Devices Affected by a Bottleneck?

Yes.

The analysis can drill down to the specific device, test case, and performance module associated with a performance problem.

The report also provides links to the corresponding performance reports.

Developers can therefore use the batch analysis to first identify the problematic devices and test cases, then open the individual reports for deeper investigation.

This creates a two-step workflow:

Batch analysis → Identify the problem → Open the detailed performance report

How Can Developers Find Crashes During Automated Testing?

Performance is not the only issue that can occur during automated testing.

A test may also encounter functional problems such as:

• Game crashes
• Unexpected exits
• Gameplay tasks that cannot be completed
• Other test failures

If a crash occurs during one of the test cases, the Test Failure Details sheet records relevant information about that test.

Developers can use information such as:

• Device
• Test time
• Runtime duration before the crash
• Memory usage at the time of the crash

as clues for further investigation.

This means that automated testing can provide more than performance data. It can also help teams identify functional failures encountered during the testing process.

What Performance Data Can Be Compared Across Test Cases?

The Performance Reports sheet provides an overview of important high-level performance metrics.

These include:

• FPS
• Memory
• Power consumption
• Temperature
• Network traffic

The report also provides scenario-level performance information, allowing developers to identify which gameplay scenarios may contain performance bottlenecks.

This makes it easier to connect a performance problem with the specific scenario in which it occurs.

When Is Batch Test Analysis Useful?

Batch Test Analysis is particularly useful for workflows that combine GOT Online with automated testing across multiple physical devices.

Compared with simply checking device compatibility, this approach provides more targeted performance information.

Instead of asking only:

"Does the game run on this device?"

developers can ask:

"Which performance problems occur on which devices, how frequently do they occur, and which problems should we optimize first?"

At the same time, crash information collected during the test provides another layer of information that development teams often need when running automated tests.

What Are the Requirements for Batch Test Analysis?

Batch Test Analysis is available for GOT Online test data generated on or after August 14.

The following conditions must be met:

• Each individual test must run for at least 3 minutes.
• At least 5 test data records are required.

When these conditions are satisfied, the Batch Test Analysis feature can be used to analyze the test batch.

What Should Teams Consider When Using Poco Automation?

When using the Poco-based automated testing scripts with GOT Online's Overview mode, the scripts may introduce additional CPU overhead.

Development teams should therefore also consider optimizing the Poco automation scripts themselves.

Otherwise, some of the CPU workload observed during the test may come from the automation process rather than the game itself.

This is particularly important when evaluating CPU-related performance metrics.

What Should Unity Teams Take Away?

• Running performance tests across many physical devices provides broader coverage but also creates a large amount of data to analyze.
• Batch analysis can identify common performance problems without requiring developers to manually inspect every report.
• Performance issues can be traced to specific devices, test cases, and performance modules.
• Crash information can be analyzed alongside performance test results.
• FPS, memory, power, temperature, and network data provide a high-level view of performance across the test batch.
• Individual performance reports can then be opened for deeper investigation.
• When using Poco-based automation, teams should account for the CPU overhead introduced by the automation scripts.

FAQ

Q1. What is Batch Test Analysis in GOT Online?

Batch Test Analysis allows developers to analyze multiple GOT Online test reports together instead of reviewing them individually. It generates an Excel workbook containing test summaries, optimization information, affected devices, test failures, and performance data.

Q2. How many test reports are required for Batch Test Analysis?

At least 5 test data records are required, and each individual test must be at least 3 minutes long.

Q3. Can Batch Test Analysis identify which devices have performance problems?

Yes. The analysis can identify specific devices, test cases, and performance modules associated with performance problems.

Q4. Can the feature detect crashes?

Yes. If a test case crashes or exits unexpectedly, the test failure information can be included in the batch analysis. Developers can use device information, test time, runtime duration, and memory usage as clues for further investigation.

Q5. What performance metrics are included in the batch analysis?

The performance overview includes high-level information such as FPS, memory, power consumption, temperature, and network traffic. Scenario-level performance information is also provided.

Q6. Does Poco automation affect performance measurements?

Potentially. Poco-based automation scripts used with GOT Online Overview mode may introduce additional CPU overhead. Teams should consider optimizing the automation scripts when interpreting CPU-related performance results.

About GameOptim GOT Online

GameOptim GOT Online is a cloud-based performance analysis platform for Unity game development. It helps development teams collect and analyze performance data across areas such as CPU, GPU, memory, power, temperature, and network performance.

With automated testing and batch analysis, teams can move from manually reviewing individual performance reports toward a more scalable performance testing workflow.