Xray Test Management for Jira is a widely used tool for test planning, execution, and reporting, offering a comprehensive set of features to manage testing workflows directly within Jira. It’s designed for teams that need full capabilities and extensive integration with Jira, making it a popular choice for larger organizations.
AgileTest by DevSamurai is also an effective test management solution that integrates with Jira. In this article, we will go through some key features in which AgileTest can be an affordable substitute for Xray Test Management for Jira.
1. Test Management
Both Xray and AgileTest support you and your team to create, manage, execute test cases, and track test results. Let’s discover how each app works.
Create and organize test cases
AgileTest
In AgileTest, there is a separate section named “Test Case”. Here’s a breakdown of the key features you can access:
- Navigation path: you can easily notice and navigate directly to see your existing test cases. With a straightforward path, you can access your entire test case library without unnecessary clicks.
- Create test cases: you could click on the button “+ Test Case” to create a test case manually. If you already have a list of test cases, you can hit the import button nearby to upload your prepared sheets.
- Organize test cases: you can drag and drop test cases into folders to categorize them into features, requirements, etc that match your preferences.
⇒ Every action you need to perform with test cases in your projects is centralized in one place, which then makes AgileTest intuitive and UI-friendly in your daily workflow.
Source: AgileTest App
XRay
In XRay, test cases are stored in the Test Repository area.
Source: XRay App
- Navigation path: you need to choose “Test Repository” → Test Folders → Test cases to see details.
- Create new test cases: the test cases in Xray are named with Test. You can add a new test case manually by clicking “Create Test”. However, if you need to import test cases, you have to move back to the menu sidebar and choose the “Test Case Importer” to upload your tests.
- Organize test cases: you can drag and drop test cases into folders for tracking and management purposes. Since one test case can only be stored in one folder, Xray allows you to categorize a test case in different Test Sets, one of its features that functions as a collection of tests.
⇒ Xray Test Management for Jira might require first-time users to spend some time getting familiar with the interface and technical naming. Despite this learning curve, Xray’s features make it an effective tool for large teams that need advanced test case management within Jira.
Source: Xray App
Execute test case
AgileTest
Before test executions, you can create a Test Plan to manage test results. The Plan helps you:
- Define test timeline: in each plan, there is a specific due date and end date. This help you track your test progress and ensure that your test execution is on the right track.
- Manage test execution: you can create and add as many test cases to each test execution as possible. Here, you will retrieve the test cases you have grouped into folders to ensure all necessary test cases are fully covered.
Source: AgileTest App
During each execution, you and your team can:
- Mark test status: you can assign a status to every test step. Then, AgileTest will calculate the test status of the overall test case. This helps you locate exactly which steps should be focused on in the next rerun.
- Add findings and comments: you can add attachments, notes, and comments during the execution sessions
- Link defects: if a bug is discovered, you can easily create a new Jira defect link or associate an existing one, enabling your tracking of both test cases and bugs in Jira.
Source: AgileTest App
⇒ AgileTest’s Test Plan feature allows you to define timelines, manage test cases, and track progress easily. During execution, you can assign statuses to individual steps, add findings, and link defects directly to Jira, making it a comprehensive yet simple solution for smaller teams or those seeking intuitive test management.
XRay
XRay’s Test Plan functions as a collection of test cases you plan to conduct for each execution session. However, there is no defined timeline to track for these plans.
- Define test timeline: there is no timeline feature to track for these plans.
- Manage test execution: you can add different test cases to a test plan. On the main page, you can see the progress for each of them.
Source: Xray App
You can conduct some actions during execution, including:
- Mark test status: you can assign a status to every test case. Xray will display the number of test cases for percentage tracking progress.
- Add findings and comments: in case you want to leave any findings, you have to add attachments and, description on the Jira ticket of the test cases.
- Link defects: if you want to link a test case with a defect ticket, you need to click on the specific test cases → Link issues → Create Defects.
Source: Xray App
⇒ Xray provides a more structured approach to test management with its Test Plan feature, allowing you to organize test cases for execution. While it lacks a built-in timeline feature and linking defects or adding findings requires navigating to Jira tickets, XRay can still be a good option for teams who are not looking for a more integrated workflow.
Report test result
AgileTest
In the Report features, AgileTest offers three main types of reports:
- Test Coverage: display how your test cases covered the predefined requirements
- Test Traceability: show the relationships among your test cases, test plans,and test executions with requirements & defects.
- Defect Summary: collect all created defects linked with your test cases during executions.
Source: AgileTest App
⇒ AgileTest reports currently are designed to give you a clear overview of your test coverage and traceability despite a limited report types.
Xray
After accessing the Reporting Center, you will see 9 types of reports for 3 main purposes:
-
Organization & Planning:
- Tests List: gather all test cases you have created
- Tests Sets List: display grouped collections of your test cases
- Tests Plans List: show all your test plans in the testing projects
<!-- /wp:list -->
-
Execution<!-- wp:list -->
- Tests Executions List: list your test executions with test status and owners' info
- Tests Runs List: summarize your individual executed test cases with details
- Test Plan Metric: overview of the test status, number of defects found, types of test cases in each test plan
<!-- /wp:list -->
-
Coverage & Analytics:<!-- wp:list -->
- Test Coverage: indicate which requirements are covered with test cases
- Requirement Traceability: show how requirements are linked with test cases, executions, and defects
- Defect Traceability: display defects and the link among them with test cases and requirements
<!-- /wp:list -->
<!-- /wp:list-item -->
Source: Xray App
⇒ Xray’s extensive reporting features provide deep insights into every aspect of your test management process, making it a great choice for larger teams that need detailed data on your testing projects.
2. Plan and Pricing
AgileTest currently has one pricing plan:
- Has a 30-day free trial
- Free for teams under 10 members
- USD 1.50 each user per month when teams are from 11 to 100 members.
Xray has two pricing plans: Standard and Advanced. The Advanced package inherits all features in the Standard package, plus a storage update from 100 to 250 MB & an increase in API calls from 60 to 100 per minute.
- Has a 30-day free trial
- Standard and Advanced Plans have a flat fee of USD 10.00 and USD 12.00 for teams of under 10 members each month
- USD 6.33 and USD 7.60 each user per month when team sizes from 11 to 100 members for the Standard and Advanced package respectively.
Final thoughts
Xray Test Management for Jira is better suited for larger teams with more complex testing workflows. It provides detailed reporting, advanced test management, and deeper Jira integration, but comes with a steeper learning curve. Meanwhile, AgileTest is a great choice for smaller teams or those looking for a simple, intuitive test management solution. It offers core features like test case creation, execution tracking, and reporting at an affordable price, making it ideal for teams that don’t need advanced customization or complex analytics. Ultimately, your choice should be based on your team's size, workflow complexity, and the level of reporting and customization you need.
AgileTest is a Jira Test Management tool that utilizes AI to help you generate test cases effectively. Try it now!
Top comments (0)