This is the last part of the specification-based testing series. Now we are going to put all of the knowledge that we gained from the other steps and start creating our tests efficiently.
If you followed every step, you already have questioned and prepared the scenarios that will help catch as many faults in your code and make sure everything is working as planned.
Let’s recap:
This is the code that we are looking forward to testing (ADD with carry flag, 8XY4 opcode from the CHIP-8 emulator):
// 8XY4
// ADD
int registerxIndex = ((instruction & 0x0F00) >> 8);
int registeryIndex = ((instruction & 0x00F0) >> 4);
int firstToSum = registers[registerxIndex];
int secondRegisterToSum = registers[registeryIndex];
int registersSum = (firstToSum + secondRegisterToSum);
registers[registerxIndex] = registersSum & 0xFF;
if (registersSum > 255) registers[registers.length - 1] = 1;
else registers[registers.length - 1] = 0;
pc += 2;
To make it easier to devise the tests, I’ll isolate this part of the code and put it in a separate class and method to make it easy to exemplify. In the real project, this is part of a CPU class that has the registers array, the program counter, etc, and a decode function to decode the instructions coming from RAM (I’ll be showing the process of creating an emulator in the future in this blog).
Here’s the simplified version of the code we are going to test:
public class AddWithCarry {
private int[] registers = new int[16];
private int pc = 0;
public void addWithCarry(int instruction) {
if( ((instruction & 0xF000)>>12) == 8 && (instruction & 0x000F) == 4) {
int registerxIndex = ((instruction & 0x0F00) >> 8);
int registeryIndex = ((instruction & 0x00F0) >> 4);
int firstToSum = registers[registerxIndex];
int secondRegisterToSum = registers[registeryIndex];
int registersSum = (firstToSum + secondRegisterToSum);
registers[registerxIndex] = registersSum & 0xFF;
if (registersSum > 255) registers[registers.length - 1] = 1;
else registers[registers.length - 1] = 0;
pc += 2;
}
}
public void setRegister(int position, int value) {
this.registers[position] = value;
}
public int getPc() {
return pc;
}
}
Step 1: Understanding the requirements
As we discussed at this step you’ll use the specification to come up with our first scenarios. By doing that we already figure out some scenarios that we can create to make sure that our code follows the behavior specified. This is mostly the successful cases.
-
Scenario 1 - Sum without carry:
- Store a value in a register X (120 for example).
- Store a value in a register Y (10 for example).
- The two values sum must not be higher than 255.
- Set the program counter to 0.
- Check if the sum result value in the register X is correct (in this scenario the sum will be 130).
- Check if the carry flag is 0.
- Check if the register Y was not changed.
- Check if the program counter was incremented by two.
-
Scenario 2 - Sum with carry:
- Store a value in a register X (235, for example).
- Store a value in a register Y (30, for example).
- Set the program counter to 0.
- The two values sum must be higher than 255.
- Check if the sum result value in the register X is correct (9 in this example). Since we had an overflow of the 255 max value, we store only the 8-bit part, since 265 is a 9-bit number (100001001), we store only the 8-bit part in X which will be 00001001 = 9.
- The 9th bit that overflows will be set in the carry flag.
- Check if the carry flag is 1.
- Check if the register Y was not changed.
- Check if the program counter was incremented by two.
-
Scenario 3 - The other registers cant affect X and Y:
- Store 0 in X and Y.
- Store values in registers that are not X and Y.
- Set the program counter to 0.
- Check if the X remains 0.
- Check if Y remains 0.
- Check if the carry is 0.
- Check if the program counter was updated by two.
Step 2: Explore the program
In the second step, we read the code. So we explored the program we found some answers about our code. The second step is for you to understand the requirements and specifications by reverse engineering your code, so you can come up with some scenarios as described above.
Step 3: Explore partitions
In the third step, we identify some scenarios that could break our code. Some possible unexpected behaviors. The key here is to focus on what’s possible to happen so we avoid over-testing scenarios that could never happen.
In this step, we found some possible unexpected inputs like:
- Scenario 1 - The instruction can be 0:
- Set instruction to 0.
- Check if register X has not changed.
- Check if register Y has not changed.
- Check if the program counter has not changed.
- Scenario 2 - Invalid instruction:
- Set instruction to an invalid value.
- Check if register X has not changed.
- Check if register Y has not changed.
- Check if the program counter did not change
In this step, we could pick some more scenarios, but to keep it short, I’ll keep these examples based only on the instruction partition. But make sure to create tests for all the other partitions of your code that can assume the expected value.
Step 4: Analyze boundaries
Last but not least, we get the scenarios we found checking the boundaries, we look at parts where one simple mistake can cause a bug with some specific values. And we found these scenarios:
- Scenario 1 -Test the exact boundary value:
- in this scenario, since we are looking at a sum, we can make that the sum of X and Y will be exactly 255.
- Check if the carry is 0.
- Scenario 2 - The boundary minus one:
- in this scenario, since we are looking at a sum, we can make that the sum of X and Y will be exactly 254.
- Check if the carry is 0.
- Scenario 3 - The boundary plus one:
- in this scenario, since we are looking at a sum, we can make that the sum of X and Y will be exactly 256.
- Check if the carry is 1.
Step 5: Devise test cases
Now we implement the scenarios that we came up with by doing the other steps of the specification-based techniques:
package org.example;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import static org.junit.jupiter.api.Assertions.*;
public class AddWithCarryTest {
@Test
public void testSumWithoutCarry() {
// Scenario 1: Sum without carry
AddWithCarry addWithCarry = new AddWithCarry();
addWithCarry.setRegister(1, 120); // Register X
addWithCarry.setRegister(2, 10); // Register Y
int instruction = 0x8124; // Add register Y to register X with carry
addWithCarry.addWithCarry(instruction);
assertEquals(130, addWithCarry.getRegisterValue(1));
assertEquals(0, addWithCarry.getRegisterValue(15)); // Carry flag should be 0
assertEquals(10, addWithCarry.getRegisterValue(2)); // Register Y should not change
assertEquals(2, addWithCarry.getPc());
}
@Test
public void testSumWithCarry() {
// Scenario 2: Sum with carry
AddWithCarry addWithCarry = new AddWithCarry();
addWithCarry.setRegister(1, 235); // Register X
addWithCarry.setRegister(2, 30); // Register Y
int instruction = 0x8124; // Add register Y to register X with carry
addWithCarry.addWithCarry(instruction);
assertEquals(9, addWithCarry.getRegisterValue(1)); // 235 + 30 = 265, only 8-bit part stored
assertEquals(1, addWithCarry.getRegisterValue(15)); // Carry flag should be 1
assertEquals(30, addWithCarry.getRegisterValue(2)); // Register Y should not change
assertEquals(2, addWithCarry.getPc());
}
@Test
public void testOtherRegistersUnaffected() {
// Scenario 3: Other registers don't affect X and Y
AddWithCarry addWithCarry = new AddWithCarry();
addWithCarry.setRegister(1, 0); // Register X
addWithCarry.setRegister(2, 0); // Register Y
addWithCarry.setRegister(3, 50); // Another register
int instruction = 0x8124; // Add register Y to register X with carry
addWithCarry.addWithCarry(instruction);
assertEquals(0, addWithCarry.getRegisterValue(1));
assertEquals(0, addWithCarry.getRegisterValue(2));
assertEquals(50, addWithCarry.getRegisterValue(3)); // Other registers should not change
assertEquals(0, addWithCarry.getRegisterValue(15)); // Carry flag should be 0
assertEquals(2, addWithCarry.getPc());
}
@Test
public void testZeroInstruction() {
// Invalid scenario: instruction is 0
AddWithCarry addWithCarry = new AddWithCarry();
addWithCarry.setRegister(1, 50);
addWithCarry.setRegister(2, 20);
int instruction = 0x0000; // Invalid instruction
addWithCarry.addWithCarry(instruction);
assertEquals(50, addWithCarry.getRegisterValue(1));
assertEquals(20, addWithCarry.getRegisterValue(2));
assertEquals(0, addWithCarry.getPc()); // PC should not change
}
@Test
public void testInvalidInstruction() {
// Invalid scenario: instruction is not added with carry
AddWithCarry addWithCarry = new AddWithCarry();
addWithCarry.setRegister(1, 50);
addWithCarry.setRegister(2, 20);
int instruction = 0x9123; // Invalid instruction for add with carry
addWithCarry.addWithCarry(instruction);
assertEquals(50, addWithCarry.getRegisterValue(1));
assertEquals(20, addWithCarry.getRegisterValue(2));
assertEquals(0, addWithCarry.getPc()); // PC should not change
}
@Test
public void testExactBoundary() {
// Boundary scenario: exact 255 sum
AddWithCarry addWithCarry = new AddWithCarry();
addWithCarry.setRegister(1, 200);
addWithCarry.setRegister(2, 55);
int instruction = 0x8124; // Add register Y to register X with carry
addWithCarry.addWithCarry(instruction);
assertEquals(255, addWithCarry.getRegisterValue(1));
assertEquals(0, addWithCarry.getRegisterValue(15)); // Carry flag should be 0
assertEquals(2, addWithCarry.getPc());
}
@Test
public void testBoundaryMinusOne() {
// Boundary scenario: sum 254
AddWithCarry addWithCarry = new AddWithCarry();
addWithCarry.setRegister(1, 200);
addWithCarry.setRegister(2, 54);
int instruction = 0x8124; // Add register Y to register X with carry
addWithCarry.addWithCarry(instruction);
assertEquals(254, addWithCarry.getRegisterValue(1));
assertEquals(0, addWithCarry.getRegisterValue(15)); // Carry flag should be 0
assertEquals(2, addWithCarry.getPc());
}
@Test
public void testBoundaryPlusOne() {
// Boundary scenario: sum 256
AddWithCarry addWithCarry = new AddWithCarry();
addWithCarry.setRegister(1, 200);
addWithCarry.setRegister(2, 56);
int instruction = 0x8124; // Add register Y to register X with carry
addWithCarry.addWithCarry(instruction);
assertEquals(0, addWithCarry.getRegisterValue(1)); // 256 wraps around to 0
assertEquals(1, addWithCarry.getRegisterValue(15)); // Carry flag should be 1
assertEquals(2, addWithCarry.getPc());
}
}
And that’s it! Now you have a good amount of scenarios that will help you guarantee the proper behavior of your code according to what was specified.
Want to know more about this topic?
In the following days, I’ll be talking about structural-based tests. Learn how to look at your code structure to come up with some test scenarios!
Stay tuned to learn more! Don't miss out!
Willian Moya (@WillianFMoya) / X (twitter.com)
Willian Ferreira Moya | LinkedIn
Follow me here on dev.to!
Top comments (0)