In the software testing, Decision table is sometimes used to cover combinations of multiple factors. This is an effective technique at various levels, such as unit testing, integration testing, functional testing, and system testing.
But if you've ever created Decision tables by hand, you know how hard it can be. Impossible conditions must be excluded from testing, and the expected results of all test cases must be determined and input by yourself.
How easy would it be if this Decision table could be automatically generated with minimal input? Furthermore, how efficient would it be if the consistency checking between those inputs was implemented at the type level by generics?
I have developed a tool tdt that satisfies the above, so I will introduce it.
Install
npm install tdt
Example
You can generate test cases as in the following example. The output format is an array of objects. One object corresponds to one test case.
const tests = generateTests(
domain,
defaults,
exclusions,
perspectives,
);
console.log(tests)
/*
[
{
'Condition.User.IsRegistered': 'True',
'Condition.User.IsAdmin': 'True',
'Condition.Device': 'Mobile',
ExpectedResult: 'Failure',
Perspective: 'Only registered users accessed from PC can access.',
ID: '1'
},
{
'Condition.User.IsRegistered': 'True',
'Condition.User.IsAdmin': 'True',
'Condition.Device': 'PC',
ExpectedResult: 'Success',
Perspective: 'Only registered users accessed from PC can access.',
ID: '2'
},
{
'Condition.User.IsRegistered': 'True',
'Condition.User.IsAdmin': 'False',
'Condition.Device': 'Mobile',
ExpectedResult: 'Failure',
Perspective: 'Only registered users accessed from PC can access.',
ID: '3'
},
{
'Condition.User.IsRegistered': 'True',
'Condition.User.IsAdmin': 'False',
'Condition.Device': 'PC',
ExpectedResult: 'Success',
Perspective: 'Only registered users accessed from PC can access.',
ID: '4'
},
{
'Condition.User.IsRegistered': 'False',
'Condition.User.IsAdmin': 'False',
'Condition.Device': 'Mobile',
ExpectedResult: 'Failure',
Perspective: 'Only registered users accessed from PC can access.',
ID: '5'
},
{
'Condition.User.IsRegistered': 'False',
'Condition.User.IsAdmin': 'False',
'Condition.Device': 'PC',
ExpectedResult: 'Failure',
Perspective: 'Only registered users accessed from PC can access.',
ID: '6'
}
]
*/
This corresponds to the decision table below. (This Markdown format table can also be output)
| #1 | #2 | #3 | #4 | #5 | #6 | ||
|---|---|---|---|---|---|---|---|
| Condition.User.IsRegistered | True | X | X | X | X | - | - |
| False | - | - | - | - | X | X | |
| Condition.User.IsAdmin | True | X | X | - | - | - | - |
| False | - | - | X | X | X | X | |
| Condition.Device | Mobile | X | - | X | - | X | - |
| PC | - | X | - | X | - | X | |
| ExpectedResult | Success | - | X | - | X | - | - |
| Failure | X | - | X | - | X | X | |
| Any | - | - | - | - | - | - |
These test cases are generic type, so missing key or value specifications can be detected before execution.
For example, the following wrong key will cause an error in static analysis such as VSCode.
let isRegistered;
isRegistered = tests[0].Conditiob.User.IsRegistered; // error
isRegistered = tests[0].Condition.Usef.IsRegistered; // error
isRegistered = tests[0].Condition.User.IsRegisteref; // error
isRegistered = tests[0].Condition.User.IsRegistered; // ok
The same is true for wrong values such as:
if(tests[0].Condition.User.IsRegistered === 'false'){ // error
if(tests[0].Condition.User.IsRegistered === 'False'){ // ok
In addition, input completion with VSCode etc. is also effective, so efficient handling is possible.
This is very useful because key names tend to be long to keep test cases readable.
Usage
To generate testcases, following inputs are required.
- Domains of the parameters (Possible values of the parameters)
- Default values of the parameters
- Combinations of the values to be excluded
- Perspectives of the testing
1. Domains of the parameters
This should include all of the possible values of all parameters related with test condition, including expected results.
This is the only input that is not generically typed, and can have any structure as long as it is an object whose terminal element is an "array of strings".
example:
const domain = {
"Condition":{
"User":{
"IsRegistered":[
"True",
"False"
],
"IsAdmin":[
"True",
"False"
]
},
"Device":[
"Mobile",
"PC"
]
},
"ExpectedResult":[
"Success",
"Failure",
"Any"
]
} as const;
type ExampleDomain = typeof domain;
2. Default values of the parameters
This should be the default values of the parameters defined in "1.". You have to choose a value from the possible values about each parameters. This is used only when the parameter is not specified in the test case.
example:
const defaults : Defaults<ExampleDomain> = {
"Condition":{
"User":{
"IsRegistered" : "True",
"IsAdmin" : "False"
},
"Device":"Mobile"
},
"ExpectedResult":"Any"
} as const;
In addition, since it is a generic type based on 1., if you specify it incorrectly, an error will occur as shown below.
3. Combinations of the values to be excluded
This is used to exclude impossible or meanless combinations of the parameter values. Those that correspond to the combination specified here are excluded from the output results.
In the example below, only one is specified, but you can specify multiple combinations.
example:
const exclusions:Exclusions<ExampleDomain> = [
{
"Condition.User.IsRegistered" : "False",
"Condition.User.IsAdmin" : "True"
}
] as const;
In addition, since it is a generic type based on 1., if you specify it incorrectly, an error will occur as shown below.
4. Perspectives of the testing
This specifies how to generate test cases. In general, all of test cases must be defined based on the perspective, what to check in testing. Therefore, all tests in this tool are generated according to specified perspectives.
-
title：Specifies the title of perspective -
constants：Specifies the factor that you want to be a constant when covering combinations. Also specify a specific value. -
variables：Specifies the factor that you want to be a variable when covering combinations. -
expect：Specifies the expected result depending on the value of each factor.
In the example, only one is specified, but you can specify multiple points of view.
example:
const perspectives:Perspectives<ExampleDomain> = [
{
"title": "Only registered users accessed from PC can access.",
"constants": {},
"variables": [
"Condition.User.IsRegistered",
"Condition.User.IsAdmin",
"Condition.Device",
],
"expect": (test:Test<ExampleDomain>)=>{
if(
test["Condition.User.IsRegistered"] === "True" &&
test["Condition.Device"] === "PC"
){
test["ExpectedResult"] = "Success";
}else{
test["ExpectedResult"] = "Failure";
}
return test
},
}
] as const;
In addition, since it is a generic type based on 1., if you specify it incorrectly, an error will occur as shown below.
Control the growth of the number of test cases
As mentioned above, this tool will automatically cover the test cases for you.
However, as the number of factors and levels of each factor increases, the number of combinations increases exponentially, and testing all cases may not be practical.
In such cases, this tool can eliminate relatively ineffective test cases by specifying strength in the concept of CoveringArray for each perspective. This keeps the total number of test cases within a feasible range.
strength is a natural number from "2" to "the number of variables". The smaller, the smaller the number of test cases. The "2" case is equivalent to the pairwise method (probably), and the "number of variables" case is exhaustive, as if not specified.
File Output
It also supports file output, and can output in JSON and markdown formats.
Future Plans
I thought about the current specifications on my own, but after that I learned about the existence of microsoft's tool PICT and the culture of TableDrivenTests. If there seems to be demand, I may change the specifications in the future to make it easier for users familiar with these.
Also, it's a test-related tool, but the problem is that it doesn't have code to test itself. I am planning to prepare in the near future.
Top comments (0)