Introduction
This whitepaper investigates the use of Ceedling frameworks by developers for Zephyr Project unit testing. It also discusses typical problems that developers run into and offers fixes.
What is Unit Testing?
Unit testing allows us to test and confirm that each module is operating and performing as intended. This facilitates the debugging process and helps identify problems early in the development phase.
The complexity and hardware dependencies of embedded systems, like those created with the Zephyr Project, make unit testing even more important.
This is a typical flowchart that shows how unit testing is done.
Advantages of Unit Testing
- Early bug detection: Identifying and addressing issues early on helps developers lower the expense and work needed in subsequent stages.
- Testing without hardware: Validation and testing individual components is possible without relying on the hardware.
- Improved code quality: Identifying issues/bugs early helps improve code quality.
- Reduced rework time: Another advantage of unit testing is that it reduces code rework time. Hence, it reduces the overall code implementation time.
- Reduces bug receive count from field: Finding as many bugs as possible at the time of development and fixing them with unit testing, improves the code quality and reduces the chances of the bugs being received from the field.
Unit Testing in the Zephyr Project
Zephyr is a real-time operating system, and it is open source. Zephyr provides the ZTest framework for unit testing.
However, in certain cases where the unit test infrastructure is built on Ceedling and the same needs to be continued for a Zephyr-based project also, this document provides guidance on some of the challenges that one might be face during the unit test implementation with Ceedling and Zephyr.
Additionally, developers often face challenges when trying to mock specific files, handle device tree contents, or simulate hardware-dependent features. This document provides solutions to these challenges using Ceedling, a popular unit testing framework. For more embedded system solutions, visit https://www.einfochips.com/
Ceedling Unit Test Framework
Overview
Ceedling is an easy to use and also widely deployed unit testing framework implemented specifically for C projects. Below are the tools integrated with Ceedling :
- Unity: It is a lightweight unit testing framework for C and useful for writing and running test cases.
- CMock: It is a mocking framework that can be used to create/generate mock functions.
- Test Reporting: It provides detailed test reports, including pass/fail results, code coverage reports, and error messages, to help developers identify and fix issues efficiently.
Ceedling is useful for embedded systems development, where testing is critical due to hardware dependency.
Key Features
- The build and test process are automated, and they use a single command to build and run the unit test cases.
- It provides a mocking functionality.
- It supports embedded projects.
Advantages of Ceedling
- It integrates Unity and CMock that simplifies the testing process. This enables automated builds and test execution. Using a single command run, all the test cases are built and run, and output reports are generated.
- It provides a mocking capability that allows one to test code without other dependencies and create the same behavior by mocking it.
- Unit tests generate detailed test reports and code coverage analysis, assisting developers in pinpointing untested code paths effectively. They also identify the function, line, and branch coverage using generated HTML reports.
- It is easy to use and widely used for the C program
How to Use Ceedling?
- Install Ceedling unit test framework using this command: gem install ceedling Ref. https://www.throwtheswitch.org/ceedling
- Create a new Ceedling project using this command: ceedling new
- Use this command to run tests cases: ceedling test:all
Common Challenges and Solutions in Unit Testing using Ceedling in Zephyr
Developers might face some challenges when writing unit test cases using Ceedling in the Zephyr project. Here are some common challenges and their solutions.
Mocking specific files without including or adding the entire zephyr framework
Challenge:
Including the entire Zephyr framework for unit testing causes the following issues:
- Increases the project size
- Increases the compilation time, and
- Introduces unnecessary dependencies that are generated at runtime
This makes the unit testing process more complex and inefficient.
Solution:
Instead of including all the Zephyr code, one needs to include or add only the required header files in project.yml file as given below:
:paths:
:include:
- <zephyr_path_where_file_exist>
For example, if kernel.h is required for testing, include a directory path where it exists and add it into the project.yml.
Include the below mock file in the test file.
#include <Mockkernel.h>
This will generate a mock file for kernel.h and allow testing without including the entire Zephyr framework.
Mocking Static Inline Functions
Challenge:
Static inline functions are replaced at the time of compilation and become part of the source file itself. As a result, CMock cannot generate mock functions for them.
Solution:
To mock static inline functions:
- Create a new header file in another custom directory to differentiate between the Zephyr header file and its directory structure.
- Include this custom directory in the project.yml file instead of the original Zephyr header file directory.
- Copy the function declaration and its dependencies into the custom header file.
- Remove the static inline keyword from the function definition.
- Include the custom header file in the test file and mock it.
- Example:
#include <Mockheader.h>
Creating Dummy Device Nodes from .dts or overlay files
Challenge:
Some device configurations, such as NVS partitions, file system partitions, or GPIO nodes, are used in the source file and need to be mocked for testing.
Solution:
Mock these configurations in the flash_map.h file as follows:
#define FIXED_PARTITION_DEVICE(partition) (void *)0 // Mock definition
Mocking Log Print Statements
Challenge:
Zephyr provides logging functions below which need to be mocked for unit testing.
- LOG_ERR,
- LOG_DBG,
- LOG_WRN, and
- LOG_INF
Solution:
Create a log.h file in a custom directory, maintaining the same Zephyr directory structure (e.g., /zephyr/logging/log.h).
Replace the Zephyr log functions with custom macros:
#define LOG_ERR(...) log_err()
#define LOG_DBG(...) log_dbg()
#define LOG_INF(...) log_inf()
#define LOG_WRN(...) log_wrn()
Mock the log.h file in the test file:
#include <Mocklog.h>
CMock will generate mock functions for the log functions:
void log_dbg(void);
void log_inf(void);
void log_wrn(void);
void log_err(void);
Mocking Functions Registered via SYS_INIT
Challenge:
The SYS_INIT macro registers initialization functions that are executed during system initialization. These functions need to be mocked for testing.
Solution:
Create an init.h file in a custom directory, maintaining the same Zephyr directory structure (e.g., /zephyr/init.h).
Replace the SYS_INIT macro with a custom implementation:
#define SYS_INIT(init_fn, level, prio) \
int custom_sys_init_fn(void) { \
int ret = init_fn(); \
return ret; \
#include <Mockinit.h>
Add the custom directory path to project.yml.
Ensure the mock header file is included in the test file to facilitate mocking and simulate dependencies effectively.
Use the custom_sys_init_fn in the test file to execute the function registered via SYS_INIT.
Including Some of Zephyr Header Files as Empty File Solely for Compilation
Challenge:
Some Zephyr header files are required for compilation but are not available during unit testing, resulting in errors like fatal error, file or directory not present.
Solution:
- Create empty versions of the required header files in a custom directory.
- Maintain the same Zephyr directory structure.
- Add the custom directory path to project.yml.
This will resolve the compilation issues.
Mocking Zephyr POSIX Header Files
Challenge:
Zephyr POSIX header files may conflict with host machine POSIX header files (e.g., timer.h).
Solution:
- Create a new file with the same name as the Zephyr header file, prefixed with an underscore (e.g., _posix_timer.h).
- Add only the required API declarations to this file.
Use this custom file for mocking.
Handling the devicetree_generated.h
Challenge:
The devicetree_generated.h file is created automatically during the build process and is generated at runtime. It is essential for unit testing but is not available by default.
Solution:
- Copy the devicetree_generated.h file from the source build folder to the test-include folder.
- Add the test-include folder path to project.yml.
- This ensures the file is available during unit testing.
By addressing these challenges with the solutions provided, developers can effectively use Ceedling to write and execute unit tests for Zephyr projects, ensuring better code quality and maintainability.
Zephyr-Ceedling Integration Example Project
To illustrate the above points, we will walk through a step-by-step example using sample code. In this demonstration, we will download Zephyr and Ceedling, and modify the hello_world sample program provided by Zephyr.
Install Zephyr and Ceedling
1. Create a Project Directory
- Create a folder named Unit_Test_Example_Project and download both Ceedling and Zephyr into this directory.
2. Set Up Zephyr
- Download Zephyr SDK and install the required dependencies. Follow the official Zephyr documentation for setup:
- Zephyr Getting Started Guide:
3. Install Ceedling
- Install Ceedling using the following command: gem install ceedling
- Refer to the official Ceedling documentation for more details:
- Ceedling Installation Guide
Note: This example uses Ceedling version 0.31.1.
Example Project Directory Structure
After setting up and modifying the Zephyr hello_world sample project, the directory structure will look as follows:
.
├── CMakeLists.txt
├── prj.conf
├── project.yml
├── rakefile.rb
├── src
│ ├── main.c
│ └── main.h
└── test
├── include
│ └── zephyr
│ ├── kernel.h
│ ├── init.h
│ ├── devicetree_generated.h
│ ├── device.h
│ ├── logging
│ │ └── log.h
│ ├── posix
│ │ └── timer.h
│ ├── storage
│ │ └── flash_map.h
└── test_main.c
Steps to Modify the ‘hello_world’ Sample Project
1. Add Missing Directories and Files
Update the Zephyr hello_world sample project by adding the directories and files shown in the directory structure above. These files and directories are not part of the original project and need to be created manually.
2. Know the Role of Each File and Folder
Before proceeding, ensure familiarity with the role of each file and folder in the project. This will help in organizing the project and writing the required source code.
3. Provide Source Code for Each File
Once the structure is in place, we provide the source code for each file in the subsequent steps.
Following these steps, one can set up a unit testing environment for the Zephyr hello_world project using Ceedling.
Directory Structure Overview
This is a Zephyr-based project that has been modified to include Ceedling for unit testing. The directory structure is organized to maintain compatibility with Zephyr's conventions while also supporting the Ceedling unit testing framework.
Top-Level Files
Source Code Directory (src)
Test Directory (test)
All the unit test related files are part of this directory. This includes the test cases, mock headers, and any additional files required for testing.
Mock Header Files in test/include
The ‘test/include’ directory contains mock or stub versions of Zephyr headers. These files are used to isolate the code under test from the actual Zephyr implementation, allowing for unit testing in a controlled environment.
Key Points
Zephyr Compatibility:
The directory structure and file organization are designed to maintain compatibility with Zephyr's build system and conventions.
The devicetree_generated.h file and other Zephyr headers are included to ensure that the test environment matches the actual application environment.
Ceedling Integration:
The test directory and project.yml file are specific to Ceedling and are used to configure and execute unit tests.
Mock headers are used to isolate the code under test from Zephyr dependencies.
Unit Test Focus:
The test_main.c file contains the actual unit test cases, which focus on testing the functions in main.c.
Mock headers simulate the behavior of Zephyr APIs, allowing the tests to run independently of the Zephyr kernel.
Conclusion
This directory structure is a modification of Zephyr’s ‘hello_world’ sample application to explain Ceedling unit testing framework in Zephyr. It allows developers to write and execute unit tests for Zephyr applications while maintaining compatibility with Zephyr's build system. The use of mock headers and the inclusion of devicetree_generated.h ensure that the test environment closely resembles the actual application environment.
CMakeLists.txt
# SPDX-License-Identifier: Apache-2.0
cmake_minimum_required(VERSION 3.20.0)
find_package(Zephyr REQUIRED HINTS $ENV{ZEPHYR_BASE})
project(Ceedling_Zephyr_Demo)
target_sources(app PRIVATE src/main.c)
prj.conf
CONFIG_LOG=y
CONFIG_LOG_DEFAULT_LEVEL=4
CONFIG_FLASH=y
CONFIG_FLASH_MAP=y
CONFIG_POSIX_API=y
CONFIG_NEWLIB_LIBC=y
Project.yml
---
:project:
:use_test_preprocessor: TRUE
:build_root: build
:test_file_prefix: test_
:which_ceedling: gem
:ceedling_version: 0.31.1
:default_tasks:
- test:all
:paths:
:test:
- +:test/**
:source:
- src/**
:include:
- src
- test/include
- ../../../zephyr/include
- ../../../zephyr/include/zephyr/fs
:defines:
:common: &common_defines
- TEST
:test:
- *common_defines
:test_preprocess:
- *common_defines
:cmock:
:mock_prefix: Mock
:enforce_strict_ordering: TRUE
:plugins:
-: ignore
- :callback
:gcov:
:reports:
- HtmlDetailed
:plugins:
:enabled:
- gcov
...
rakefile.rb
require "ceedling"
Ceedling.load_project
task :default => %w[test:all]
src/main.c
/**
* @file main.c
*
* @brief This file contains function implementations to demonstrate challenges with Ceedling and Zephyr,
* and provides solutions in unit testing code.
*
* @author Meetkumar Bariya
*
* @date 2025-06-15
*
*/
#include "main.h"
#include <zephyr/device.h>
#include <zephyr/init.h>
#include <zephyr/kernel.h>
#include <zephyr/logging/log.h>
#include <zephyr/storage/flash_map.h>
#include <sys/types.h>
#include <stdbool.h>
#include <time.h>
LOG_MODULE_REGISTER(main);
#define STORAGE_PARTITION storage_partition
#define STORAGE_DEVICE FIXED_PARTITION_DEVICE(STORAGE_PARTITION)
#define STORAGE_OFFSET FIXED_PARTITION_OFFSET(STORAGE_PARTITION)
static int initialize_system(void);
/**
* @brief System initializes at boot-up time.
*
* @return int Return up-time.
*/
static int initialize_system(void)
{
LOG_INF("System initialization started.");
k_sleep(K_SECONDS(1)); // Point 2: Example of static inline function.
int up_time = k_uptime_get(); // Point 2: Example of static inline function.
LOG_INF("System initialization completed successfully.");
return up_time;
}
/**
* @brief Demonstrates the use of POSIX time functions.
*/
struct timespec demonstrate_posix_time(void)
{
struct timespec current_time = {0, 0};
if (clock_gettime(CLOCK_REALTIME, ¤t_time) == 0) { // Point 7: POSIX function
LOG_INF("Current time retrieved successfully: %lld seconds, %ld nanoseconds.",
current_time.tv_sec, current_time.tv_nsec);
}
return current_time; // Return the current time
}
/**
* @brief Check flash device ready or not.
*/
void verify_flash_device(void)
{
if (!device_is_ready(STORAGE_DEVICE)) {
LOG_ERR("flash device has been not ready. Please check the configuration.");
return;
}
LOG_INF("Flash storage device is ready for use.");
}
/* Register the system initialization function to run during application startup. */
SYS_INIT(initialize_system, APPLICATION, CONFIG_APPLICATION_INIT_PRIORITY); // Point 5: SYS_INIT
src/main.h
/**
* @file main.h
*
* @brief This file includes the function prototypes of main.c file.
*
* @author Meetkumar Bariya
*
* @date 2025-06-15
*
*/
#ifndef MAIN_H
#define MAIN_H
/**
* @brief This function was created to demonstrate POSIX APIs.
*
* This function retrieves the current system time using POSIX-compliant
*
* `clock_gettime` and logs the result. It is useful for validating time-related
*
* functionality in the system.
*
* Returns the current time.
*/
struct timespec demonstrate_posix_time(void);
/**
* @brief Verifies the readiness of the flash storage device.
*
* This function checks whether the flash storage device is ready for use.
* If the device is not ready, an error message is logged. Otherwise, a success
*
* message is logged to indicate that the device is operational.
*/
void verify_flash_device(void);
#endif /* MAIN_H */
test_main.c
/**
* @file test_main.c
*
* @brief Unit test file for `main.c`, demonstrating the use of Ceedling with mocked dependencies.
* This file contains tests for system initialization, flash device verification, and POSIX time functionality.
*
* The tests validate the behavior of the functions in `main.c` by mocking external dependencies
* such as logging, device readiness checks, kernel functions(static inline), and POSIX time functions.
*
* @author Meetkumar Bariya
*
* @date 2025-06-15
*
* @details This test file uses the Unity framework and Ceedling to perform unit testing. Mocked dependencies
* are used to isolate the functions under test and ensure their correctness in various scenarios.
*
* The below features are included:
* - System initialization (`initialize_system`): Validates logging, kernel sleep time, kernel uptime and return value.
* - Flash device verification (`verify_flash_device`): Ensures readiness checks and logging behavior.
* - POSIX time demonstration (`demonstrate_posix_time`): Validates time retrieval and logging behavior.
*
* Mocked dependencies:
* - `Mockdevice.h`: Mocks device readiness checks.
* - `Mockkernel.h`: Mocks kernel functions like `k_sleep` and `k_uptime_get`.
* - `Mocklog.h`: Mocks logging functions.
* - `Mocktimer.h`: Mocks POSIX time functions like `clock_gettime`.
*/
#include "unity.h"
#include "main.h"
#include "Mockdevice.h" // Mocking device readiness checks
#include "Mockkernel.h" // Mocking kernel functions like k_sleep and k_uptime_get
#include "Mocklog.h" // Mocking logging functions
#include "Mocktimer.h" // Mocking POSIX time functions
#include <stdbool.h>
#include <stddef.h>
#include <sys/types.h>
extern bool custom_sys_init_fn(void);
void setUp(void)
{
// Initialize any required resources before each test
}
void tearDown(void)
{
// Clean up work can be added here.
}
/**
* @brief Stub function for `clock_gettime` used in unit testing.
*
* Simulates the behavior of `clock_gettime` by validating the clock ID and setting mock time values.
*
* @param[in] clk_id Expected to be `CLOCK_REALTIME`.
* @param[out] tp Pointer to a `timespec` structure to store mock time values.
* @param[in] cmock_num_calls Number of times the stub has been called.
*
* @return int Returns `0` to indicate success.
*/
static int clock_gettime_stub(clockid_t clk_id, struct timespec *tp, int cmock_num_calls)
{
// Validate that the clock ID is CLOCK_REALTIME
TEST_ASSERT_EQUAL(CLOCK_REALTIME, clk_id);
// Set the mock values
tp->tv_sec = 12345;
tp->tv_nsec = 67890;
return 0;
}
/**
* @brief Test the system initialization function.
*
* This test function is to validate the SYS_INIT function:
* - Logs the appropriate messages.
* - Calls the `k_sleep` and `k_uptime_get` functions.
* - Returns the expected success value.
*/
void test_initialize_system(void)
{
// Expect the mock log function for the "System initialization started" log print
log_inf_Expect();
// Mock the static inline function
k_sleep_ExpectAndReturn(K_SECONDS(1), 0); // Expect k_sleep to return 0
int64_t expected_up_time = 10; // Mocked up-time value
k_uptime_get_ExpectAndReturn(expected_up_time); // Mock the k_uptime_get function to return a fixed value
// Expect the mock log function for the "System initialization completed successfully" log print
log_inf_Expect();
// Simulate the SYS_INIT behavior and call the function to test ('initialize_system')
int64_t result = custom_sys_init_fn();
TEST_ASSERT_EQUAL(expected_up_time, result);
}
/**
* @brief This function is to test the flash device.
*
* This test validates that the `verify_flash_device` function:
* - Checks the readiness of the flash device.
* - Logs appropriate messages based on the device's readiness.
*/
void test_verify_flash_device_function(void)
{
// (1) Success case: Device is ready
// Mock the device_is_ready function
device_is_ready_ExpectAndReturn(NULL, true);
log_inf_Expect();
// Call the function that needs to test
verify_flash_device();
// (2) Failure case: Device is not ready
// Mock the device_is_ready function
device_is_ready_ExpectAndReturn(NULL, false);
log_err_Expect();
verify_flash_device();
}
/**
* @brief Test the POSIX time demonstration function.
*
* This test validates that the ‘demonstrate_posix_time’ function:
*
*/
void test_demonstrate_posix_time(void)
{
// Stub the clock_gettime function
clock_gettime_StubWithCallback(clock_gettime_stub);
log_inf_Expect();
struct timespec expected_time = { .tv_sec = 12345, .tv_nsec = 67890 };
struct timespec actual_time = demonstrate_posix_time();
// Validate the seconds and nanoseconds
TEST_ASSERT_EQUAL(expected_time.tv_sec, actual_time.tv_sec);
TEST_ASSERT_EQUAL(expected_time.tv_nsec, actual_time.tv_nsec);
}
device.h
#ifndef CUSTOM_DEVICE_H
#define CUSTOM_DEVICE_H
#include <stdbool.h>
bool device_is_ready(const void *dev);
#endif /* CUSTOM_DEVICE_H */
devicetree_generated.h
Copy the content from ‘Unit_Test_Example_Project/zephyrproject/zephyr/samples/hello_world/build/zephyr/include/generated/zephyr/devicetree_generated.h’.
Note: If this file is not present, then build the source code using the command below, so that it is generated.
west build -b esp32c3_devkitm -p
init.h
#ifndef CUSTOM_INIT_H
#define CUSTOM_INIT_H
#define SYS_INIT(init_fn, level, prio) \
int custom_sys_init_fn (void) { \
int ret = init_fn(); \
return ret; \
}
#endif /* CUSTOM_INIT_H */
Kernel.h
#ifndef CUSTOM_KERNEL_H
#define CUSTOM_KERNEL_H
#include <stdint.h>
typedef int64_t k_ticks_t;
typedef struct {
k_ticks_t ticks;
} k_timeout_t;
#define K_SECONDS (sec) ((k_timeout_t) { .ticks = (sec * 1000) })
int32_t k_sleep(k_timeout_t timeout);
int64_t k_uptime_get(void);
#endif // CUSTOM_KERNEL_H
log.h
#ifndef CUSTOM_LOG_H
#define CUSTOM_LOG_H
#define LOG_MODULE_REGISTER(name)
void log_dbg(void);
void log_inf(void);
void log_wrn(void);
void log_err(void);
#define LOG_ERR(...) log_err()
#define LOG_DBG(...) log_dbg()
#define LOG_INF(...) log_inf()
#define LOG_WRN(...) log_wrn()
#endif /* CUSTOM_LOG_H */
timer.h
#ifndef CUSTOM_TIMER_H
#define CUSTOM_TIMER_H
#include <time.h>
#define CLOCK_REALTIME 0 // Mock value for CLOCK_REALTIME
int clock_gettime(int clock_id, struct timespec *tp);
#endif /* CUSTOM_TIMER_H */
flash_map.h
#ifndef CUSTOM_STORAGE_FLASH_MAP_H
#define CUSTOM_STORAGE_FLASH_MAP_H
#define FIXED_PARTITION_DEVICE(partition) (void *)0 // Mock definition
#endif // CUSTOM_STORAGE_FLASH_MAP_H
Build Commands
west build -b <board_name> -p
i.e. west build -b esp32c3_devkitm -p
To build and run test code,
ceedling gcov:all utils:gcov --clobber
Coverage Reports
Generated coverage report will be generated at ‘samples/hello_world/build/artifacts/gcov/ GcovCoverageResults.html’ location.
About The Author
Meetkumar Bariya is a Senior Software Engineer with 10 years of experience in embedded systems and real-time operating systems. He is currently working at eInfochips. Meetkumar has a bachelor's degree in Electronics and Communication Engineering from Ganpat University, and an MBA degree in IT Management from Symbiosis college, Pune.
Top comments (0)