DEV Community: Leo Liu

Autonomous Obstacle Avoidance System for UAVs Based on OMAPL138 DSP+ARM+FPGA, Millimeter-Wave Radar, and Monocular Vision Fusion

Leo Liu — Wed, 10 Jun 2026 07:25:53 +0000

Originally published on the Sienovo Engineering Blog. Sienovo is the overseas brand of 深圳信迈 (Shenzhen Xinmai), building edge AI computing solutions for industrial video analytics.

In recent years, civil small unmanned aerial vehicles (UAVs) have received increasing attention and application, becoming widely used in fields such as express delivery, food delivery, and power line inspection. UAVs equipped with autonomous obstacle avoidance capabilities hold broad application prospects in social production, scientific research, and aerial photography operations. Such systems not only ensure stable flight but also enable UAVs to effectively avoid obstacles during complex missions, reducing damage from collisions and associated secondary hazards. Scholars worldwide have conducted extensive research on UAV autonomous obstacle avoidance. Conventional UAV obstacle avoidance solutions typically rely on pre-existing 3D elevation maps, multi-camera visual sensors, or combinations of multiple sensors. However, existing obstacle avoidance systems generally suffer from drawbacks such as large size, high power consumption, heavy weight, and high cost. Autonomous obstacle avoidance systems that are compact, low-power, lightweight, and low-cost—suitable for installation on small UAVs such as multirotor drones—remain an area requiring further research.

Millimeter-wave radar offers advantages including small size, light weight, and long-distance ranging capability, but it suffers from low accuracy in measuring obstacle azimuth angles. Monocular vision sensors can estimate the azimuth angles of obstacles in the UAV's flight environment and, compared to multi-camera systems, require less computational effort. However, monocular vision performs poorly in measuring obstacle distance. Integrating millimeter-wave radar with a monocular vision sensor on a small UAV enables the acquisition of high-precision obstacle distance and azimuth angle information, achieving complementary strengths between the two sensors. Moreover, the computational load of this solution is low, making it suitable for general embedded microprocessors and ideal for deployment on small UAVs.

Based on this UAV autonomous obstacle avoidance approach, the main research contributions of this paper are as follows:

1) A review of the current state of research and development in UAV autonomous obstacle avoidance systems is presented, followed by the proposal of a fusion-based system combining millimeter-wave radar and monocular vision sensors.

2) The architecture of millimeter-wave radar and its fundamental principles for obstacle distance measurement are introduced. Methods for applying digital image processing theory to monocular vision-based obstacle angle estimation are investigated. A method for fusing millimeter-wave radar distance measurements with monocular vision angle measurements to achieve 3D obstacle localization is proposed. Additionally, a technique for simplifying complex 3D path planning into 2D path planning is discussed.

3) Based on the Sienovo OMAPL138 DSP+ARM+FPGA hardware platform, a millimeter-wave radar platform, and the DJI Naza UAV platform, a demonstration system for UAV autonomous obstacle avoidance using millimeter-wave radar and monocular vision fusion was developed. Obstacle avoidance flight experiments were conducted using this system, providing preliminary validation of the feasibility of the proposed fusion approach in UAV autonomous flight.

Evaluation Board Overview Based on the TI OMAP-L138 (fixed/floating-point DSP C674x + ARM9) and Xilinx Spartan-6 FPGA processor; OMAP-L138 and FPGA are connected via uPP, EMIFA, and I2C buses, supporting communication speeds up to 228 MByte/s; OMAP-L138 operates at a maximum clock frequency of 456 MHz, delivering up to 3648 MIPS and 2746 MFLOPS computational performance; FPGA supports Xilinx Spartan-6 XC6SLX9/16/25/45, offering strong platform scalability; The board exposes abundant peripherals, including Gigabit Ethernet, SATA, EMIFA, uPP, USB 2.0, and other high-speed data interfaces, as well as common interfaces such as GPIO, I2C, RS232, PWM, and McBSP; Certified for high and low temperature operation, suitable for harsh working environments; The DSP+ARM+FPGA tri-core SOM measures 66mm × 38.6mm and uses industrial-grade B2B connectors to ensure signal integrity; Supports bare-metal, SYS/BIOS, and Linux operating systems.

Figure 1 Front and side views of the evaluation board

The XM138F-IDK-V3.0 is a development board designed based on the Sienovo XM138-SP6-SOM core module, fabricated using a 4-layer lead-free ENIG process. It provides a testing platform for the XM138-SP6-SOM core module, enabling rapid evaluation of its overall performance.

The XM138-SP6-SOM exposes all CPU resource signal pins, making secondary development extremely easy. Customers can focus on upper-layer applications, significantly reducing development difficulty and time-to-market, allowing products to quickly enter the market and gain competitive advantage. In addition to rich demo programs, comprehensive development tutorials and full technical support are provided to assist customers with carrier board design, debugging, and software development.

Typical Application Areas Data acquisition, processing, and display systems Smart power systems Image processing equipment High-precision instrumentation Mid-to-high-end CNC systems Communication equipment Audio and video data processing

Figure 2 Typical application areas

Introduction to OMAP-L138 + FPGA Development Board The XM138F-IDK-V3, designed by Sienovo, is a high-speed data acquisition and processing development board based on a DSP+ARM+FPGA tri-core architecture, suitable for applications in power systems, communications, industrial control, medical devices, and audio/video processing.

This design employs the OMAP-L138 + Spartan-6 platform. The OMAP-L138 is a low-power, high-performance dual-core processor from Texas Instruments (TI), integrating a floating-point DSP C6748 and an ARM9 core. The Spartan-6 is a Xilinx FPGA processor known for flexible platform scalability and excellent cost-performance ratio. The two chips are interconnected via communication interfaces such as uPP and EMIF on the OMAP-L138, while the internal DSP and ARM cores communicate via DSPLINK/SYSLINK, forming a unique, flexible, and powerful DSP+ARM+FPGA tri-core high-speed data acquisition and processing system.

Resource Block Diagram of OMAP-L138 + FPGA Development Board

Figure 1 Block diagram of the OMAP-L138 + FPGA tri-core high-speed data acquisition and processing system

System Architecture Description:

The front end uses a Xilinx Spartan-6 XC6SLX9/16/25/45 FPGA to synchronously acquire two-channel AD data with a maximum sampling rate of 65 MHz. The AD data is transmitted to the OMAP-L138's DSP via the uPP or EMIF bus.
After processing by the DSP, data is transferred to the ARM via DSPLINK or SYSLINK inter-processor communication components for application interface development, network forwarding, SATA disk storage, and other applications.
Based on processing results, the OMAP-L138's DSP or ARM sends logic control commands to the FPGA, which drives the onboard DA to generate logic outputs with an update rate of 175 MSPS.

(1) High-speed data acquisition front end: The Xilinx Spartan-6 XC6SLX9/16/25/45 FPGA synchronously acquires two analog input signals and can perform pre-filtering on the AD data, with a maximum sampling rate of 65 MSPS. Another DAC channel outputs parallel DA data with arbitrary amplitude and waveform, updated at 175 MSPS.

(2) High-speed data transmission: Implemented via uPP, EMIF, SPI, and I2C buses. High-throughput AD and DA data are stably transferred between the DSP and FPGA via the uPP bus. The DSP controls FPGA logic and performs medium-throughput data exchange via the EMIF bus. The ARM configures and initializes the FPGA via SPI and I2C.

(3) High-speed data processing: Composed of the DSP core and algorithm libraries, enabling real-time signal processing such as FFT and FIR filtering for time-domain, frequency-domain, and amplitude analysis of AD and DA data.

(4) DSP+ARM dual-core communication: Composed of the DSP core, ARM core, and DSPLINK/SYSLINK inter-processor communication components. Data exchange between the DSP and ARM is achieved via shared memory.

(5) Data display and storage expansion: Composed of the ARM core, graphics display, network, and SATA disk. The ARM-based application interface enables real-time display of time-domain and frequency-domain waveforms of AD and DA signals, as well as large-scale data storage and remote network communication.

Xilinx Spartan-6 FPGA High-Speed Data Acquisition Front-End Logic Implementation

Xilinx Spartan-6 FPGA — XC6SLXx

The XC6SLXx is a widely used, cost-effective FPGA platform from the Xilinx Spartan-6 series, featuring 324 pins, abundant available I/Os, good platform scalability, reasonable cost differences across variants, and smooth scalability. Depending on application needs, models such as LX9, LX16, LX25, and LX45 can be selected. Additionally, these four CPU models are pin-to-pin compatible.

(1) XC6SLX9: Interface-level, capable of interface programming and clock control.

(2) XC6SLX16: Algorithm-level, suitable for simple algorithm processing.

(3) XC6SLX25: Algorithm-level, capable of intermediate-level algorithm processing.

(4) XC6SLX45: System-level, meeting requirements for complex algorithms and system logic processing.

High-Speed ADC — AD9238

The AD9238 is a 12-bit dual-channel A/D converter from Analog Devices (ADI), offering the industry's fastest sampling rates. It supports selectable input voltage ranges of 1 Vp-p and 2 Vp-p and is widely used in high-speed data acquisition applications in power, communications, industrial control, and medical systems.

The AD9238 comes in three variants with sampling rates of 20 MS/s, 40 MS/s, and 65 MS/s. It delivers dynamic performance comparable to single-channel ADCs and offers better crosstalk immunity than using two separate single-channel ADCs. The three variants are pin-to-pin compatible, enabling flexible configuration based on requirements.

The power consumption of the three AD9238 variants is 180 mW, 330 mW, and 600 mW, respectively—about half that of comparable ADCs. Packaged in a 64-pin LQFP (9 mm × 9 mm), it is ideal for space-constrained applications.

High-Speed DAC — AD9706

The AD9706 is a 12-bit D/A converter from Analog Devices with a 175 MSPS update rate and output current range of 1 mA to 5 mA, widely used in high-speed data output applications in communications, industrial control, medical, and power systems.

/// All are 175 MSPS update-rate D/A converters, pin-to-pin compatible, with resolutions of 8, 10, 12, and 14 bits. These devices are optimized for low power while maintaining excellent dynamic performance, support flexible supply voltage ranges (1.7 V to 3.6 V), and are ideal for portable and low-power applications. Power consumption can be reduced to 15 mW by lowering full-scale output current, and standby power in power-down mode can be as low as 2.2 mW.

The AD9748/AD9740/AD9742/AD9744 series D/A converters are also pin-to-pin compatible (in LFCSP_VQ package) and optimized for transmit signal paths in communication systems. Users can select suitable devices based on performance, resolution, and cost requirements.

Implementation of Communication Between Xilinx Spartan-6 FPGA and TI OMAP-L138

Figure 6 FPGA-OMAP-L138 Communication Principle

High-Speed Communication Bus — uPP

The uPP (Universal Parallel Port) is a distinctive high-speed parallel data transfer interface on the OMAP-L138 CPU. It supports independent or simultaneous data transmission and reception, commonly used for data exchange with FPGAs and other parallel-interface devices.

The OMAP-L138 uPP features two channels (Channel A and Channel B), 32 data lines, simple control, flexible configuration, and high data throughput. The uPP clock rate can reach up to half the processor clock rate. For the OMAP-L138 running at 456 MHz, the theoretical single-channel throughput can reach up to 228 MB/s.

Implementation of DSP and ARM Dual-Core Communication on TI OMAP-L138

Figure 14 OMAP-L138 DSP+ARM Dual-Core Communication Principle

Basic Principle

TI's official DSPLINK/SYSLINK dual-core communication components provide a set of generic APIs that abstract the physical connection characteristics between ARM and DSP at the application layer, reducing development complexity. DSPLINK operates with the DSP/BIOS OS, while SYSLINK uses the SYS/BIOS OS; SYSLINK is the newer version of the DSPLINK dual-core communication component.

In dual-core development, the ARM runs a high-level operating system (HLOS), typically Linux, while the DSP runs a real-time operating system (RTOS), usually DSP/BIOS or SYS/BIOS. Both cores operate at 456 MHz.

Advantages

(1) The on-chip DSP+ARM architecture enables stable dual-core communication, significantly reducing development time.

(2) The DSPLINK/SYSLINK components overcome dual-core development bottlenecks, reducing R&D costs.

(3) The integration of DSP and ARM on a single SOC simplifies hardware design, lowering product power consumption and hardware costs.

This article was translated from Chinese to English with AI assistance and a light human review. The original is published at Sienovo Blog. The original Chinese source is at CSDN. Learn more about Sienovo edge AI computing.

Adapter Pattern and Facade Pattern (Highly Recommended)

Leo Liu — Tue, 09 Jun 2026 07:11:13 +0000

Originally published on the Sienovo Engineering Blog. Sienovo is the overseas brand of 深圳信迈 (Shenzhen Xinmai), building edge AI computing solutions for industrial video analytics.

Original Link: 适配器模式和外观模式

Adapter Pattern (Adapter)

Let's start with an introduction and a problem. As is well known, the common voltage in China is 220V, while in the United States it is 110V. If an IT professional frequently travels between the US and China, always carrying their laptop, how can the laptop's voltage issue be resolved? (Because the voltage differs between the US and China, general electrical appliances are not interchangeable.) This is where the adapter shines.

Modern laptops all have a power adapter, and it is precisely this power adapter that solves the aforementioned adaptation problem. For example, a Sony laptop might have an input voltage range of AC 100V~240V, with a unified DC output of 19.5V. AC current is input into one end of the power adapter, which then converts the power to the required voltage. In essence, the adapter's role is to make one thing suitable for another.

Here is the definition of the Adapter Pattern

The Adapter Pattern converts the interface of a class into another interface clients expect.

It is mainly used in the following situations:

For example, suppose you have an old software system with an outdated component that needs updating. You acquire a third-party component (the new component), but its interface differs from the old component's. At the same time, you don't want to change the existing code (which might be impossible if the system is large). This is where the Adapter Pattern comes in. You can use the Adapter Pattern to convert some interfaces of the new component into the interfaces you expect (i.e., compatible with the old component). This way, you can easily update from the old component to the new one without altering the original code.

Another good application is when many things that run on Windows cannot run on Linux. For instance, a tool like WINE allows users to run Windows programs in a Linux environment; this is also a form of adapter.

In essence, the Adapter Pattern can be understood as wrapping some objects and making their interfaces appear as different interfaces.

It's also worth mentioning: The Adapter Pattern is traditionally divided into Class Adapter Pattern and Object Adapter Pattern. However, since the Class Adapter Pattern relies on multiple inheritance, and C# does not support multiple inheritance, only the Object Adapter is introduced here. If you are interested in the Class Adapter Pattern, you can implement it using C++.

First, let's look at the Target class (the Target class represents the interface that clients can use)

namespace Adapter 
{ 
    public abstract class Target 
    { 
        // Temperature 
        /// <summary> 
        /// The interfaces below are the ones recognizable by the client, i.e., the target interfaces. 
        /// The Chinese methods in the Adaptee class, however, are not recognizable by the client and need to be adapted. 
        /// </summary> 
        public abstract void GetTemperature();

        // Pressure 
        public abstract void GetPressure();

        // Humidity 
        public abstract void GetHumidity();

        // UV Intensity 
        public abstract void GetUltraviolet(); 
    } 
}

Next, let's look at the Adaptee class (the interfaces in Adaptee are not recognizable by the client, so they need to be adapted)

using System;

namespace Adapter 
{ 
    class Adaptee 
    { 
        /// <summary> 
        /// The interfaces in the Adaptee class are not the ones the client needs. 
        /// For example, here Chinese is used, but the client requires English. 
        /// Therefore, an adapter must be used here. 
        /// </summary> 
        public void 得到温度() 
        { 
            Console.WriteLine("You got today's temperature"); 
        }

        public void 得到气压() 
        { 
            Console.WriteLine("You got today's pressure"); 
        }

        public void 得到湿度() 
        { 
            Console.WriteLine("You got today's humidity"); 
        }

        public void 得到紫外线强度() 
        { 
            Console.WriteLine("You got today's UV intensity"); 
        } 
    } 
}

Then, let's look at the code for the Adapter (the Adapter indirectly converts interfaces not recognizable by the client into recognizable ones)

namespace Adapter 
{ 
    public class Adapter:Target 
    { 
        // An Adaptee object must be maintained within the adapter 
        private Adaptee adaptee = new Adaptee();

        /// <summary> 
        /// Adapt interfaces not originally recognized by the client through the adapter 
        /// </summary> 
        public override void GetTemperature() 
        { 
            adaptee.得到温度(); 
        }

        public override void GetPressure() 
        { 
            adaptee.得到气压(); 
        }

        public override void GetHumidity() 
        { 
            adaptee.得到湿度(); 
        }

        public override void GetUltraviolet() 
        { 
            adaptee.得到紫外线强度(); 
        } 
    } 
}

Finally, let's look at the client code

using System;

namespace AdapterTest 
{ 
    class Program 
    { 
        static void Main(string[] args) 
        { 
            // Instantiate an adapter for the target interface 
            Adapter.Target target = new Adapter.Adapter();

            // The following are the interfaces recognizable by the client 

            target.GetTemperature(); 
            target.GetPressure(); 
            target.GetHumidity(); 
            target.GetUltraviolet();

            Console.ReadKey(); 
        } 
    } 
}

===================================================================

Facade Pattern (Facade)

Let's start with an example from 'Head First Design Patterns' (I've reorganized that example). The example describes how many people in the US set up home theaters (I'll consider the simplest approach, which is just turning everything on and off). In a home theater, you typically need basic tools like lights, a screen, a projector, an amplifier, and a DVD player. Lights can be turned on or off. The projector can be turned on or off. The screen can also be opened or closed. The amplifier can have its volume turned up or down. The DVD player can be turned on or off.

Then, to watch a movie, I would have to perform the following operations on the client side: first turn on the projector, then turn on the amplifier, then open the screen, then turn on the DVD player, and finally turn on the lights. After all these operations, you can finally watch a movie (how inconvenient, so many steps, too complicated!). And when you're done, you still have to turn off the projector, then the amplifier, then close the screen, then turn off the DVD player, and finally turn off the lights. Oh, this is too complicated!!!

There are so many operations on the client side (and the problem is that some users might not know how to use one of these tools, meaning they can't watch a movie). Users would be utterly annoyed!!!

This actually reflects a common phenomenon in modern software development systems: client programs often have direct contact with the internal subsystems of a complex system, causing the client program to change as the subsystems change. In the example above, the client program is the user's operation, and the internal subsystems of the complex system represent the usage interfaces of these tools.

In the example above, I only used the simplest tool operation interfaces, namely simple on/off functions. What if there are new features in the subsystems, i.e., in each tool? This would inevitably lead to changes in the client code.

To solve this series of problems, you must simplify the interaction interface between the client program and the subsystems (to enable users who don't know how to use all the tools to still watch a movie). Then, you need to decouple the client program from the subsystems, and the Facade Pattern can solve this problem perfectly.

Definition of Facade Pattern (Facade)

The Facade Pattern provides a unified interface to a set of interfaces in a subsystem. This pattern defines a higher-level interface that makes the subsystem easier to use.

Simply put, the Facade Pattern hides the complex operations of one or more classes, exposing only a consistent interface for the client to use.

It's also important to note that the Facade Pattern merely provides you with a more direct and easier way to operate; it does not isolate the original subsystems. Therefore, if you still need higher-level functionality from the subsystem classes, you can still use the original subsystems. This is a major advantage of the Facade Pattern.

At the same time, the Facade Pattern allows for the creation of a high-level interface over multiple subsystem interfaces, providing this high-level interface to the client. This decouples the client from the complex subsystems.

Furthermore, the Facade Pattern also enables us to adhere to the Law of Demeter (also known as the Principle of Least Knowledge).

Now let's look at the code for a few playback tools

using System;

namespace Facade 
{ 
    /// <summary> 
    /// Projector 
    /// </summary> 
    public class Projector 
    { 
        public void OpenProjector() 
        { 
            Console.WriteLine("Open projector"); 
        }

        public void CloseProjector() 
        { 
            Console.WriteLine("Close projector"); 
        }

        public void SetWideScreen() 
        { 
            Console.WriteLine("Projector is in widescreen mode"); 
        }

        public void SetStandardScreen() 
        { 
            Console.WriteLine("Projector is in standard mode"); 
        } 
    } 
}

using System;

namespace Facade 
{ 
    /// <summary> 
    /// Amplifier 
    /// </summary> 
    public class Amplifier 
    { 
        public void OpenAmplifier() 
        { 
            Console.WriteLine("Open amplifier"); 
        }

        public void CloseAmplifier() 
        { 
            Console.WriteLine("Close amplifier"); 
        } 
    } 
}

using System;

namespace Facade 
{ 
    /// <summary> 
    /// Screen 
    /// </summary> 
    public class Screen 
    { 
        public void OpenScreen() 
        { 
            Console.WriteLine("Open screen"); 
        }

        public void CloseScreen() 
        { 
            Console.WriteLine("Close screen"); 
        } 
    } 
}

using System;

namespace Facade 
{ 
    /// <summary> 
    /// DVD Player 
    /// </summary> 
    public class DVDPlayer 
    { 
        public void OpenDVDPlayer() 
        { 
            Console.WriteLine("Open DVD player"); 
        }

        public void CloseDVDPlayer() 
        { 
            Console.WriteLine("Close DVD player"); 
        } 
    } 
}

using System;

namespace Facade 
{ 
    /// <summary> 
    /// Light 
    /// </summary> 
    public class Light 
    { 
        public void OpenLight() 
        { 
            Console.WriteLine("Turn on light"); 
        }

        public void CloseLight() 
        { 
            Console.WriteLine("Turn off light"); 
        } 
    } 
}

Next, here's the code for the Facade class

namespace Facade 
{ 
    /// <summary> 
    /// Define a facade 
    /// </summary> 
    public class MovieFacade 
    { 
        /// <summary> 
        /// The facade class must hold references to various objects in the subsystem 
        /// </summary> 
        private Projector projector; 
        private Amplifier amplifier; 
        private Screen screen; 
        private DVDPlayer dvdPlayer; 
        private Light light;

        public MovieFacade() 
        { 
            projector = new Projector(); 
            amplifier = new Amplifier(); 
            screen = new Screen(); 
            dvdPlayer = new DVDPlayer(); 
            light = new Light(); 
        }

        /// <summary> 
        /// Open movie 
        /// </summary> 
        public void OpenMovie() 
        { 
            // First, turn on the projector 
            projector.OpenProjector(); 
            // Then, turn on the amplifier 
            amplifier.OpenAmplifier(); 
            // Then, open the screen 
            screen.OpenScreen(); 
            // Then, turn on the DVD 
            dvdPlayer.OpenDVDPlayer(); 
            // Then, turn on the lights 
            light.OpenLight(); 
        }

        /// <summary> 
        /// Close movie 
        /// </summary> 
        public void CloseMovie() 
        { 
            // Turn off the projector 
            projector.CloseProjector(); 
            // Turn off the amplifier 
            amplifier.CloseAmplifier(); 
            // Close the screen 
            screen.CloseScreen(); 
            // Turn off the DVD 
            dvdPlayer.CloseDVDPlayer(); 
            // Turn off the lights 
            light.CloseLight(); 
        } 
    } 
}

Finally, here's the client code

using System;

namespace FacadeTest 
{ 
    class Program 
    { 
        static void Main(string[] args) 
        { 
            Facade.MovieFacade movie = new Facade.MovieFacade(); 
            Facade.Projector projector = new Facade.Projector();

            // First, watch the movie 
            movie.OpenMovie();

            Console.WriteLine();

            // Then, set the projector to widescreen mode 
            projector.SetWideScreen(); 
            // Then, set the projector back to standard mode 
            projector.SetStandardScreen(); 
            Console.WriteLine();

            // Finally, close the movie 
            movie.CloseMovie();

            Console.ReadKey(); 
        } 
    } 
}

As can be seen from the screenshots above, I can still use the content from the subsystem in the client. That is, the Facade Pattern does not isolate the subsystem from the client; it merely provides a clean interface to the client. However, if the client wants to access interfaces within the complex subsystem, it can still do so, as demonstrated in the Demo above where the projector within the subsystem was accessed and

Microcontroller Communication Protocol Design

Leo Liu — Mon, 08 Jun 2026 07:37:31 +0000

Originally published on the Sienovo Engineering Blog. Sienovo is the overseas brand of 深圳信迈 (Shenzhen Xinmai), building edge AI computing solutions for industrial video analytics.

Most modern instruments require operation through host computer software for convenient debugging and ease of use. This inherently involves communication. Based on several devices he has developed, the author has summarized a general approach to writing communication programs, covering both host and slave ends.

　　1. Custom Data Communication Protocol

　　The data protocol discussed here refers to the format of communication data packets built upon the physical layer. The physical layer of communication refers to common methods such as RS232, RS485, infrared, fiber optic, wireless, and so on. At this layer, the low-level software provides two basic operations: sending a byte of data and receiving a byte of data. All data protocols are built upon these two operations.

Data in communication is often transmitted in the form of data packets, which we refer to as a data frame. Similar to TCP/IP protocols in network communication, a relatively reliable communication protocol typically includes the following components: frame header, address information, data type, data length, data block, checksum, and frame tail.

　　The frame header and frame tail are used to determine the completeness of the data packet. They usually consist of a fixed length of specific bytes. The requirement is that the bit error rate for identifying data packets in the entire data link should be as low as possible. This means minimizing the chance of matching the characteristic bytes of the frame header and frame tail within the entire data stream. There are generally two approaches: first, reduce the probability of matching characteristic bytes; second, increase the length of characteristic bytes. The first method is usually chosen when the data in the entire data link is not random and is predictable, allowing the characteristic words for the frame header and frame tail to be artificially selected to avoid matches, thereby reducing the probability of matching characteristic bytes. The second method is more general and suitable for situations where data is random. Increasing the length of characteristic bytes reduces the matching probability, and although it cannot completely avoid matching, it significantly lowers the chances. If a match does occur, it can also be detected by the checksum. Therefore, this approach is quite reliable in most cases.

　　Address information is primarily used in multi-device communication to identify different communication terminals based on their unique addresses. In a one-to-many communication system, it may only include destination address information. Including both source and destination addresses is suitable for many-to-many communication systems.

　　Data type, data length, and data block constitute the main data part. The data type identifies whether the subsequent data is a command or actual data. The data length indicates the number of valid data bytes.

　　The checksum is used to verify the integrity and correctness of the data. It is typically derived from calculations involving the data type, data length, and data block. The simplest approach is to perform a cumulative sum on the data segment, while more complex methods might involve CRC calculations, etc., chosen based on requirements for calculation speed, fault tolerance, and other factors.

　　2. Data Transmission in Host and Slave

　　The physical communication layer provides two basic operational functions, and sending a single byte of data forms the basis of data transmission. Sending a data packet simply involves transmitting all bytes within the packet one by one in sequence. Of course, there are different methods for transmission.

　　In microcontroller systems, a common method is to directly call a serial function that sends a single byte of data. The disadvantage of this method is that the processor needs to be fully involved throughout the transmission process. The advantage is that the data to be sent appears immediately on the communication line and can be received by the receiving end without delay. Another method is to use interrupt-driven transmission, where all data to be sent is placed into a buffer, and a transmit interrupt sends the data from the buffer. The advantage of this method is that it consumes fewer processor resources, but there might be a slight delay before the data is actually sent, though this delay is usually very small. For 51-series microcontrollers, direct transmission is often preferred, as interrupt-driven transmission consumes more RAM resources and doesn't offer significant advantages over direct transmission. Below is a function for sending a single byte in a 51-series microcontroller.

void SendByte(unsigned char ch)  
{  
     SBUF = ch;  
     while(TI == 0);  
     TI = 0;  
}

　　There are also various methods for serial communication on the host computer. This refers not to whether data is buffered, but to different ways of operating the serial port, as data transmission on a PC is almost always buffered before sending. For programming, there are three ways to operate a serial port: First, use the serial communication controls built into the Windows system. This method is relatively simple, but attention must be paid to blocking handling during reception and threading mechanisms. Second, use system APIs directly to read serial data. In Windows and Linux systems, devices are virtualized as files, and serial data can be sent and received using the API functions provided by the system. Third, use a serial port class for serial operations. Here, we will only introduce programming with a serial port class in a Windows environment.

　　CSerialPort is a commonly used serial port class. It provides the following serial port operation method:

　　void WriteToPort(char* string, int len);

　　After the serial port is successfully initialized, this function can be called to send data to the serial port. To avoid delays caused by serial port buffering, the serial port's flush mechanism can be enabled.

　　3. Data Reception and Protocol Parsing in Slave

　　There are also two ways for the slave to receive data: first, wait for reception, where the processor continuously queries the serial port status to determine if data has been received; second, interrupt-driven reception. The advantages and disadvantages of these two methods were discussed in detail in a previous article on serial communication. The conclusion was that interrupt-driven reception is generally better.

　　The data packet parsing process can be placed in different locations. If the protocol is simple and the entire system only handles simple commands, the data packet parsing process can be directly integrated into the interrupt service routine. When a correct data packet is received, the corresponding flag is set, and the main program then processes the command. If the protocol is slightly more complex, a better approach is to store the received data in a buffer, and the main program reads and parses the data later. There are also hybrid approaches, for example, in a one-to-many system, the "connect" command is first parsed in the receive interrupt. After the connect command is received, the main program enters a setup state and uses polling to parse the remaining protocols.

　　A specific example is provided below. In this system, the serial port commands are very simple. All protocols are handled within the serial interrupt. The data packet format is as follows:

　　0x55, 0xAA, 0x7E, 0x12, 0xF0, 0x02, 0x23, 0x45, SUM, XOR, 0x0D

　　Here, 0x55, 0xAA, 0x7E are the frame header, 0x0D is the frame tail, 0x12 is the device's destination address, 0xF0 is the source address, 0x02 is the data length, followed by two data bytes 0x23, 0x45. The cumulative sum and XOR checksum are calculated starting from the destination address and ending at the last data byte.

　　The purpose of protocol parsing is first to determine the completeness and correctness of the data packet, then to extract data type, data, and other information, and store it for processing by the main program. The code is as follows:

if(state_machine == 0)       // Protocol parsing state machine  
{  
      if(rcvdat == 0x55)      // Received first byte of frame header  
          state_machine = 1;  
      else  
          state_machine = 0;      // Reset state machine  
}  
else if(state_machine == 1)  
{  
      if(rcvdat == 0xAA)      // Received second byte of frame header  
          state_machine = 2;  
      else  
          state_machine = 0;      // Reset state machine  
}  
else if(state_machine == 2)  
{  
      if(rcvdat == 0x7E)      // Received third byte of frame header  
          state_machine = 3;  
     else  
          state_machine = 0;      // Reset state machine  
}  
else if(state_machine == 3)  
{  
      sumchkm = rcvdat;      // Start calculating cumulative sum and XOR checksum  
      xorchkm = rcvdat;  
      if(rcvdat == m_SrcAdr)      // Check if destination address is correct  
          state_machine = 4;  
      else  
          state_machine = 0;  
}  
else if(state_machine == 4)  
{  
      sumchkm += rcvdat;  
      xorchkm ^= rcvdat;  
      if(rcvdat == m_DstAdr)      // Check if source address is correct  
          state_machine = 5;  
      else  
          state_machine = 0;  
   }  
else if(state_machine == 5)  
{  
      lencnt = 0;             // Data reception counter  
      rcvcount = rcvdat;          // Received data length  
      sumchkm += rcvdat;  
      xorchkm ^= rcvdat;  
      state_machine = 6;  
}  
else if(state_machine == 6 || state_machine == 7)  
{  
      m_ucData[lencnt++] = rcvdat;    // Save data  
      sumchkm += rcvdat;  
      xorchkm ^= rcvdat;  
      if(lencnt == rcvcount)      // Check if data reception is complete  
          state_machine = 8;  
      else  
          state_machine = 7;  
}  
else if(state_machine == 8)  
{  
      if(sumchkm == rcvdat)      // Check if cumulative sum matches  
          state_machine = 9;  
      else  
          state_machine = 0;  
}  
else if(state_machine == 9)  
{  
      if(xorchkm == rcvdat)      // Check if XOR checksum matches  
          state_machine = 10;  
      else  
          state_machine = 0;  
}  
else if(state_machine == 10)  
{  
      if(0x0D == rcvdat)          // Check if frame tail terminator is received  
      {  
          retval = 0xaa;      // Set flag, indicating a data packet is received  
      }  
      state_machine = 0;        // Reset state machine  
}

　　During this process, a variable state_machine is used as the protocol state machine's transition state to determine which part of a data frame the current byte belongs to. Simultaneously, checksum calculation and processing are performed automatically during reception. When the data packet reception is complete, the checksums are compared. Therefore, when the frame tail terminator is received, it indicates that a data frame has been fully received and validated, and the key data has been saved to the buffer. The main program can then use the retval flag to proceed with protocol parsing.

　　During reception, if the data received at any step is not the expected value, the state machine is immediately reset to prepare for the next data frame. This significantly reduces the occurrence of state deadlocks, making the system relatively stable. If data packets are lost, the host can re-send commands, though the author has not encountered such a situation.

　　The process for protocol handling in the main program is similar. The main program continuously reads data from the serial buffer in its loop, and this data is then fed into the protocol processing routine within the main loop. The code is exactly the same as described above.

　　4. Data Reception and Command Processing in Host

　　The data reception process on the host can be entirely consistent with that on the slave, though there are differences depending on the serial port operation method. For blocking serial read functions, such as direct API operations or calling Windows serial communication controls, it is best to start a dedicated thread to monitor serial data reception. Each time a data byte is received, a message can be sent to the system. The CSerialPort class, commonly used by the author, handles this process in a similar way. After CSerialPort opens the serial port, it starts a thread to monitor serial data reception, saves the received data to a buffer, and sends a data reception message to the parent process. The data is sent along with the message to the parent process. The parent process enables the message handling function, and after obtaining serial data from it, the code described above can be copied and used.

　　The message ID sent by CSerialPort to the parent class is as follows:

　　#define WM_COMM_RXCHAR WM_USER+7 // A character was received and placed in the input buffer.

　　Therefore, a response function for this message needs to be manually added:

　　afx_msg LONG OnCommunication(WPARAM ch, LPARAM port);

　　ON_MESSAGE(WM_COMM_RXCHAR, OnCommunication)

　　The specific code for the response function is as follows:

LONG CWellInfoView::OnCommunication(WPARAM ch, LPARAM port)  
{   
     int retval = 0;  
     rcvdat = (BYTE)ch;

     if(state_machine == 0)      // Protocol parsing state machine  
    {  
        if(rcvdat == 0x55)      // Received first byte of frame header  
            state_machine = 1;  
        else  
            state_machine = 0;      // Reset state machine  
    }  
    else if(state_machine == 1)  
    {  
        if(rcvdat == 0xAA)      // Received second byte of frame header  
            state_machine = 2;  
        else  
            state_machine = 0;      // Reset state machine  
    }  
    ......

　　5. Conclusion

　　The above provides a basic prototype for how a communication system operates. Although simple, it is feasible. Actual communication systems have more complex protocols and involve a series of issues such as data packet responses, command errors, and delays. Building upon this foundation, these difficulties can be overcome to achieve a more stable and reliable system.

ARM+FPGA-based Industrial Medical Endoscope Solution

Leo Liu — Sun, 07 Jun 2026 07:21:21 +0000

Originally published on the Sienovo Engineering Blog. Sienovo is the overseas brand of 深圳信迈 (Shenzhen Xinmai), building edge AI computing solutions for industrial video analytics.

Introduction

This article describes the first installment of a multi-part series on an FPGA-based monocular endoscope positioning system for cardiac surgery simulation. The system uses a camera, SDRAM frame buffering, and real-time image processing on an FPGA to track and localize the tip of a surgical catheter inside a cardiac simulator — giving trainee surgeons coordinate feedback without requiring direct visual inspection of the heart interior. This post covers the project background, FPGA fundamentals, system design rationale, algorithm selection, and hardware module overview.

1. Background and Motivation

1.1 Medical Device Context

Cardiovascular disease treatment is among the most commercially and clinically significant segments of the medical device industry. Cardiac assist devices — whether implantable or external, electronic or mechanical — have steadily decreased in size, making them easier to implant. At the same time, cardiac electrophysiology has emerged as a rapidly evolving subspecialty: it studies the electrical phenomena generated by excitable tissue (action potentials in nerves, cardiac muscle, etc.) and uses that information to guide interventional procedures.

A critical bottleneck in expanding surgical capacity is the shortage of experienced surgeons. Current estimates suggest that only about 6% of cardiac patients who could benefit from surgery actually receive it; many others die waiting, or survive with unmanaged risk. The root cause is that training a competent cardiac surgeon through traditional apprenticeship — operating on real patients under supervision — is slow, expensive, and carries ethical and medico-legal risk, particularly in the current climate of heightened doctor-patient tension. A cardiac surgery simulator that lets trainees practice catheter navigation in a realistic phantom heart, while receiving real-time position feedback, directly addresses this bottleneck.

Image processing is the enabling technology here. The system described in this series captures live endoscope video from inside a cardiac simulator, detects the catheter tip in each frame, outputs its 2-D coordinates for display, and forwards contact-point signals to an external instrument for 3-D cardiac modeling.

1.2 FPGA in Medical Imaging

FPGAs (Field-Programmable Gate Arrays) have long been the preferred platform for real-time video processing because their massively parallel datapath architecture maps naturally to pipeline-oriented image algorithms. In a typical FPGA image pipeline, each processing stage operates on a pixel as it arrives from the sensor, so the end-to-end latency is a handful of clock cycles rather than a full frame period. This makes sustained frame rates of 25 fps straightforward to achieve, with 60 fps attainable on mid-range devices. Early FPGA adoption was concentrated in communications (baseband coding, modulation/demodulation), but the technology has migrated into medical imaging, ECG processing, endoscopy, and other clinical signal-processing domains. The design described here targets catheter-tip localization, but the same FPGA platform can be extended to multi-camera configurations for 3-D coordinate recovery.

2. FPGA Architecture and Design Flow

2.1 How FPGAs Implement Logic

An FPGA achieves programmability through three cooperating mechanisms:

Programmable logic blocks. The fundamental element is a Look-Up Table (LUT) — typically 4-input or 6-input, 1-output — implemented in SRAM or Flash. A 4-input LUT can store the truth table of any 4-input/1-output combinational function; changing the stored bits reconfigures the function. Paired with a D flip-flop, any sequential logic can be expressed as combinational logic plus register, making the LUT+FF pair a universal building block.

Programmable interconnect. Individual LUT+FF cells are too small to be useful alone. A mesh of programmable routing resources — segments, switch matrices, and connection boxes controlled by configuration bits — joins thousands of cells into large, arbitrary logic networks. Asserting or de-asserting configuration bits connects or disconnects individual wire segments, giving the designer full control over the netlisting.

Programmable I/O. Every FPGA pin (except dedicated power, ground, and programming pins) can be configured as input, output, or bidirectional. Voltage standard, drive strength, slew rate, and on-chip termination are all software-settable, allowing the device to interface to a wide range of external components without external level-shifting hardware.

2.2 Design Flow

A typical FPGA design moves through the following stages:

RTL entry — HDL source (VHDL or Verilog) is written or generated from IP cores, or captured schematically for small blocks.
Functional simulation — logic behavior is verified in a simulator before synthesis. ModelSim is the most widely used tool for this stage, valued for its simulation speed and accuracy. Cadence Verilog-XL and Synopsys VCS are also common in larger teams; ActiveHDL adds a state-machine debugger that is useful for control-path verification.
Synthesis — the HDL is compiled to a gate-level netlist optimized for area and timing. Synopsys Synplify/Synplify Pro is widely used for its timing-driven synthesis engine and fast runtimes. Xilinx XST (now Vivado Synthesis) and the integrated synthesizer in Altera/Intel Quartus II are the primary vendor alternatives.
Implementation (map + place + route) — the synthesized netlist is mapped to device primitives, placed within the FPGA fabric, and routed. This stage is performed by vendor tools: Xilinx ISE/Vivado or Intel Quartus II.
Static timing analysis — the place-and-route tools report critical paths and maximum achievable clock frequency. The design constraint is that logic utilization must remain below 80% of available macrocells to preserve routability margin.
Post-route simulation — gate-level simulation with extracted back-annotation verifies that timing-dependent behavior matches intent.
Bitstream generation and download — a configuration bitstream is generated and programmed into the FPGA (or companion Flash/EEPROM for non-volatile retention).

3. System Design

3.1 Design Task

The goal is to detect the 2-D position of a catheter tip as it moves within the field of view of an endoscope camera. The catheter enters the camera's field of view from any edge of the frame; only the tip coordinate is of interest — the rest of the catheter body is ignored. The FPGA implementation must keep logic utilization below 80% of available macrocells.

3.2 Algorithm Selection

Three candidate detection strategies were evaluated:

Option A — Color-based segmentation. The catheter tip is painted a distinctive color; the FPGA isolates the tip by thresholding a specific color component or HSV range. This approach is simple but requires pre-processing the catheter (painting it) and is sensitive to background clutter with similar colors. Rejected.

Option B — Frame difference method. Consecutive frames are subtracted; the absolute-value difference image is thresholded to yield a binary motion mask, highlighting the moving catheter while suppressing the static background. The method is insensitive to lighting and background complexity because it only responds to temporal change. Its weaknesses are sensitivity to high-frequency noise (requiring a denoising pre-stage) and parameter sensitivity: if the inter-frame interval is too short relative to catheter speed, two separated tip blobs appear; if too long relative to a slow-moving catheter, no motion is detected. The resulting binary image shows edges rather than a solid region, requiring frame buffering for correct processing. Selected for this design.

Option C — Background subtraction. A clean background frame is stored and subtracted from each live frame; the residual is thresholded to isolate the catheter. The method produces a solid binary region (better fill than frame difference), but the stored background becomes stale as lighting or scene conditions change. Compensating for this requires a background-update algorithm (mean, median, Kalman filter, or Gaussian mixture), which adds considerable FPGA complexity. Implementing a robust background update in hardware is the primary difficulty; this option would have been viable but was judged harder to implement reliably within the logic budget.

The decision matrix is summarized in Table 3.1 (see original figures). Option B (frame difference) was selected based on its combination of acceptable fill quality, low background-complexity requirement, and manageable FPGA implementation effort.

3.3 Key Technical Challenges

Seven problems were identified as technically critical:

Image frame buffering in SDRAM
Image pre-processing (noise removal)
Frame-difference computation
Ping-pong SDRAM buffer management
Pixel format conversion chain
Binary image projection
Correct catheter-tip coordinate extraction from projection data

3.4 Solutions

Frame buffering. The design requires three simultaneous SDRAM accesses: the camera writes incoming pixels to one of two alternating regions, while two read ports simultaneously deliver both buffered frames to the subtraction stage. A dual-buffered SDRAM interface is used, with two FIFOs on the write side and two on the read side, satisfying the three-port concurrency requirement.

Image pre-processing. Morphological operations are applied before differencing: erosion removes isolated noise pixels, and dilation expands the remaining foreground regions to fill gaps and strengthen connectivity.

Frame differencing. The two buffered frames are subtracted pixel-by-pixel; the absolute value of the difference is taken (|Frame_N − Frame_{N−1}|), eliminating dependence on subtraction order.

Ping-pong buffer switching. The camera write alternates between buffer 0 and buffer 1 on successive frames; the read side mirrors this assignment one frame behind. A common mistake is using the capture or output module's "done" signal to trigger the buffer switch — this can cause data accumulated in the FIFOs to be flushed by a reset before it is read, resulting in partial frame loss. The correct approach is to use the SDRAM controller's internal completion signal to drive the switch, as implemented here.

Format conversion. The OV7670 outputs YUV 4:2:2 (YCbCr422). The pipeline converts this to YUV444, then to RGB888 (using a lookup table for the YCbCr→RGB matrix), and finally to RGB565 for VGA output.

Binary image projection. After thresholding the difference image to binary, horizontal and vertical projections are computed: the horizontal projection sums pixel values along each row (Y axis), and the vertical projection sums along each column (X axis). The resulting 1-D profiles are stored and used to find the bounding box of the motion region.

Tip coordinate extraction. The four boundary lines derived from the projection profiles define the bounding rectangle of the detected motion region. Within that rectangle, the algorithm computes the midpoint of the horizontal boundary span (endpoint 7 in Figure 3.1), then compares the distance from endpoint 7 to the right boundary against the distance from endpoint 7 to the left boundary to determine catheter orientation. Points lying on the boundaries are classified as non-target; the outlier point not on any boundary is identified as the catheter tip, and its coordinates are output.

4. Hardware Module Overview

4.1 Power Module

The system requires multiple supply rails: 5 V (VCC), 3.3 V, 2.8 V, 2.5 V, and 1.2 V. All are generated from the AMS1117 family of low-dropout linear regulators. AMS1117 devices are available in fixed-output variants (1.2 V, 1.5 V, 1.8 V, 2.5 V, 2.8 V, 3.0 V, 3.3 V) and an adjustable variant. Each regulator includes built-in thermal shutdown and current-limit protection. A 22 µF output capacitor is placed on each regulator output to ensure stability and suppress high-frequency transients.

4.2 Acquisition Module (OV7670 Camera)

The OV7670 CMOS image sensor is selected for its wide availability, well-documented register interface, and straightforward FPGA integration. Key interface signals are:

SCCB / I²C bus — used to configure internal registers (resolution, format, exposure, white balance, etc.)
XCLK — master clock input to the sensor
PCLK — pixel clock output; runs at twice the pixel rate to allow 8-bit parallel data capture
FRAME_VALID / LINE_VALID — frame and line synchronization signals
DOUT[7:0] — 8-bit parallel pixel data output

Additional signals include STROBE (flash sync), STANDBY/PWDN (power management), and reset. The sensor output is configured for YUV 4:2:2 format.

4.3 Buffer Module (SDRAM)

The selected SDRAM is the Hynix H57V2562GTR: 256 Mb (16 M × 16 bit), stable operation to 200 MHz, 4 banks, 13-bit row address, 16-bit data width. It stores the YCbCr 4:2:2 data stream from the camera. In the video capture context, the SDRAM is configured for page-write burst mode with burst-interrupt support, enabling arbitrary burst lengths and maximizing sustained bandwidth — essential for maintaining real-time frame rates in the processing pipeline.

4.4 Display Module (VGA)

VGA is chosen as the display output for its universality: virtually all monitors and projectors support VGA, and the interface is straightforward to implement in FPGA logic (R/G/B analog signals driven by simple resistor-ladder DACs, plus HSYNC and VSYNC). VGA supports high resolutions, fast refresh rates, and full color depth. Limitations — no hot-plug detection, no embedded audio — are not relevant for this application.

What's Next

This first installment has covered the project motivation, FPGA architecture fundamentals, algorithm trade-offs, key implementation challenges, and the top-level hardware block diagram. The next installment will present the detailed hardware design: power supply schematics, FPGA peripheral circuits, acquisition circuit, SDRAM buffer circuit, and VGA output circuit. The following installment will then cover the full software (RTL) implementation of each module — acquisition, buffering, frame-difference processing, format decoding, and display — along with simulation waveforms and board-level test results.

Virtual Functions in C++

Leo Liu — Sat, 06 Jun 2026 07:02:09 +0000

Originally published on the Sienovo Engineering Blog. Sienovo is the overseas brand of 深圳信迈 (Shenzhen Xinmai), building edge AI computing solutions for industrial video analytics.

Virtual Functions in C++ (1)

Virtual functions are a mechanism in C++ used to achieve polymorphism. The core idea is to access functions defined in derived classes through a base class.

I. Introduction
Virtual functions are a mechanism in C++ used to achieve polymorphism. The core idea is to access functions defined in derived classes through a base class. Suppose we have the following class hierarchy:

class A
{
public:
virtual void foo() { cout << "A::foo() is called" << endl;}
};
class B: public A
{
public:
virtual void foo() { cout << "B::foo() is called" << endl;}
};

Then, when using it, we can:

A * a = new B();
a->foo(); // Here, although 'a' is a pointer to A, the function called (foo) is B's!

This example is a typical application of virtual functions, and through it, you might already have some concept of what they are. Their "virtual" nature lies in what's called "deferred binding" or "dynamic binding," where the call to a class function is not determined at compile time but at runtime. Because it's not possible to determine at the time of writing the code whether the base class's function or a derived class's function will be called, they are referred to as "virtual" functions.

Virtual functions can only achieve polymorphism through pointers or references. If the code is as follows, even though it's a virtual function, it won't exhibit polymorphism:

class A
{
public:
virtual void foo();
};
class B: public A
{
virtual void foo();
};
void bar()
{
A a;
a.foo(); // A::foo() is called
}

1.1 Polymorphism

After understanding what virtual functions mean, it's easy to grasp polymorphism. Still considering the class hierarchy above, but with a slightly more complex usage:

void bar(A * a)
{
a->foo(); // Is A::foo() or B::foo() called?
}

Because foo() is a virtual function, within the bar function, based solely on this code, it's impossible to determine whether A::foo() or B::foo() will be called. However, it can be stated with certainty: if a points to an instance of class A, then A::foo() is called; if a points to an instance of class B, then B::foo() is called.
This characteristic, where the same code can produce different effects, is called "polymorphism."
1.2 What is polymorphism used for?

Polymorphism is amazing, but what can it be used for? This is a question I find hard to summarize in a sentence or two. Most C++ tutorials (or tutorials for other object-oriented languages) use a drawing example to demonstrate the use of polymorphism, so I won't repeat that example. If you don't know it, any book should cover it. I'll try to describe it from an abstract perspective, and then, by revisiting the drawing example, it might be easier for you to understand.
In object-oriented programming, abstraction (determining base classes) and inheritance (determining derived classes) are first performed on data to form a class hierarchy. If users of this class hierarchy still write code specific to the base class when a base class is needed, and code specific to a derived class when a derived class is needed, it means the class hierarchy is completely exposed to the user. If there are any changes to this class hierarchy (e.g., adding a new class), the user needs to "know" about it (write code for the new class). This increases the coupling between the class hierarchy and its users, which some consider one of the "bad smells" in programming.
Polymorphism can relieve programmers from this predicament. Looking back at the example in 1.1, bar() as a user of the A-B class hierarchy doesn't know how many classes are in this hierarchy or what each class is called, yet it can still work well. When a class C is derived from class A, bar() doesn't need to "know" (be modified). This is entirely due to polymorphism—the compiler generates code for virtual functions that can determine the called function at runtime.
1.3 How "Dynamic Binding" Works

How does the compiler generate code for virtual functions that can determine the called function at runtime? In other words, how are virtual functions actually processed by the compiler? Lippman discusses several methods in different chapters of "Inside the C++ Object Model" [1]. Here, I'll briefly introduce the "standard" method.
What I refer to as the "standard" method is the so-called "VTABLE" mechanism. When the compiler finds a function declared as virtual in a class, it creates a virtual function table for it, known as a VTABLE. A VTABLE is essentially an array of function pointers, with each virtual function occupying a slot in this array. A class has only one VTABLE, regardless of how many instances it has. Derived classes have their own VTABLEs, but the derived class's VTABLE has the same function arrangement order as the base class's VTABLE; virtual functions with the same name are placed in the same position in both arrays. When creating a class instance, the compiler also adds a vptr field to the memory layout of each instance, which points to the class's VTABLE. Through these means, when the compiler sees a virtual function call, it rewrites this call. For the example in 1.1:

void bar(A * a)
{
a->foo();
}

would be rewritten as:

void bar(A * a)
{
(a->vptr[1])();
}

Because the foo() functions of derived and base classes have the same VTABLE index, and their vptrs point to different VTABLEs, this method allows determining which foo() function to call at runtime.
Although the actual situation is far more complex, the basic principle is roughly as described.
1.4 Overload and Override

Virtual functions are always rewritten in derived classes; this rewriting is called "override." I often confuse the words "overload" and "override." However, with the increasing number of C++ books, later programmers may no longer make the mistakes I did. But I intend to clarify:
override refers to a derived class rewriting a virtual function of its base class, just as our class B rewrote the foo() function in class A. The rewritten function must have a consistent parameter list and return type (the C++ standard allows for different return types in some cases, which I will briefly introduce in the "Syntax" section, but few compilers support this feature). This word doesn't seem to have a suitable Chinese equivalent; some translate it as "覆盖" (cover), which is somewhat apt.
overload is conventionally translated as "重载." It refers to writing a function with the same name as an existing function but with a different parameter list. For example, a function that can accept an integer as a parameter and also a floating-point number as a parameter.

II. Virtual Function Syntax

The hallmark of a virtual function is the "virtual" keyword.
2.1 Using the virtual keyword

Consider the following class hierarchy:

class A
{
public:
virtual void foo();
};
class B: public A
{
public:
void foo(); // No virtual keyword!
};
class C: public B // Inherits from B, not A!
{
public:
void foo(); // Also no virtual keyword!
};

In this case, B::foo() is a virtual function, and C::foo() is also a virtual function. Therefore, it can be said that a virtual function declared in a base class remains virtual in derived classes, even if the virtual keyword is no longer used.
2.2 Pure Virtual Functions

The following declaration indicates a function is a pure virtual function:

class A
{
public:
virtual void foo()=0; // =0 marks a virtual function as pure virtual
};

Once a function is declared pure virtual, it means: "I am an abstract class! Do not instantiate me!" Pure virtual functions are used to standardize the behavior of derived classes, essentially serving as an "interface." They tell the user that all my derived classes will have this function.
2.3 Virtual Destructors

Destructors can also be virtual, or even pure virtual. For example:

class A
{
public:
virtual ~A()=0; // Pure virtual destructor
};

When a class is intended to be used as a base class for other classes, its destructor must be virtual. Consider the following example:

class A
{
public:
A() { ptra_ = new char[10];}
~A() { delete[] ptra_;} // Non-virtual destructor
private:
char * ptra_;
};
class B: public A
{
public:
B() { ptrb_ = new char[20];}
~B() { delete[] ptrb_;}
private:
char * ptrb_;
};
void foo()
{
A * a = new B;
delete a;
}

In this example, the program might not run as you expect. When delete a is executed, only A::~A() is actually called, and the destructor of class B is not called! Isn't that a bit scary?
If A::~A() above is changed to virtual, it can guarantee that B::~B() is also called when delete a. Therefore, base class destructors must be virtual.
A pure virtual destructor serves no particular purpose; being virtual is sufficient. Usually, a pure virtual destructor is used only when you want to make a class abstract (a class that cannot be instantiated) and there is no other suitable function to make pure virtual.
2.4 Virtual Constructors?
Constructors cannot be virtual.

III. Virtual Function Usage Tips
3.1 Private Virtual Functions

Consider the following example:

class A
{
public:
void foo() { bar();}
private:
virtual void bar() { ...}
};
class B: public A
{
private:
virtual void bar() { ...}
};

In this example, even though bar() is private in class A, it can still appear in derived classes and still produce polymorphic effects just like public or protected virtual functions. It won't happen that A::foo() cannot access B::bar() because it's private, nor will B::bar()'s override of A::bar() be ineffective.
The semantic meaning of this writing style is: A tells B, "You'd better override my bar() function, but don't worry about how it's used, and don't call this function yourself."
3.2 Virtual Function Calls in Constructors and Destructors

When a class's virtual function is called within its own constructor or destructor, they become ordinary functions and are no longer "virtual." This means you cannot achieve polymorphism within constructors and destructors. For example:

class A
{
public:
A() { foo();} // Here, A::foo() is always called, no matter what!
~A() { foo();} // Same as above
virtual void foo();
};
class B: public A
{
public:
virtual void foo();
};
void bar()
{
A * a = new B;
delete a;
}

If you expect delete a to cause B::foo() to be called, then you are mistaken. Similarly, when new B is executed, A's constructor is called, but within A's constructor, A::foo() is called, not B::foo().
3.3 Virtual Functions in Multiple Inheritance 3.4 When to use virtual functions

When designing a base class, if you find that a function needs to behave differently in derived classes, then it should be virtual. From a design perspective, a virtual function in a base class is an interface, and a virtual function in a derived class is a concrete implementation of that interface. Through this method, the behavior of objects can be abstracted.
Taking the Factory Method pattern [2] in design patterns as an example, the factoryMethod() of the Creator is a virtual function. After derived classes override this function, they produce different Product classes, and the generated Product classes are used by the base class's AnOperation() function. The base class's AnOperation() function operates on the Product class, and naturally, the Product class must also have polymorphism (virtual functions).
Another example is collection operations. Suppose you have a class hierarchy with class A as the base class, and you use a std::vector to store pointers to instances of different classes in this hierarchy. You would certainly want to operate on the classes in this collection without having to cast each pointer back to its original type (derived class), but rather perform the same operation on all of them. In this case, that "same operation" should be declared virtual.
In reality, there are far more examples than the two I've given, but the general principle is what I stated earlier: "if you find that a function needs to behave differently in derived classes, then it should be virtual." This statement can also be reversed: "if you find that a base class provides a virtual function, then you'd better override it."

Appendix: Usage of Virtual Functions and Pure Virtual Functions in C++

Virtual functions and pure virtual functions can be defined in the same class. A class containing pure virtual functions is called an abstract class, while a class containing only virtual functions cannot be called an abstract class.
A virtual function can be used directly or called polymorphically after being overridden by a subclass. A pure virtual function must be implemented in a subclass before it can be used, because a pure virtual function only has a declaration but no definition in the base class.
Both virtual functions and pure virtual functions can be overridden in subclasses and called polymorphically.
Virtual functions and pure virtual functions typically exist in abstract base classes (ABC) and are overridden by inherited subclasses to provide a unified interface.
The definition form of a virtual function is: virtual {method body}; the definition form of a pure virtual function is: virtual { } = 0;. The static identifier cannot be used in the definition of virtual functions and pure virtual functions. The reason is simple: functions modified by static require early binding at compile time, whereas virtual functions are dynamically bound (run-time bind), and the lifetime of functions modified by both is also different.
If a class contains pure virtual functions, any attempt to instantiate that class will result in an error, because abstract base classes (ABC) cannot be called directly. They must be inherited and overridden by subclasses, and their subclass methods called as required. The following is a simple demonstration of virtual and pure virtual function usage, intended to spark further thought!

#include
//father class
class Virtualbase
{
public:
virtual void Demon()= 0; //prue virtual function
virtual void Base() {cout<<"this is farther class"<};
//sub class
class SubVirtual :public Virtualbase
{
public:
void Demon() { cout<<" this is SubVirtual!"< void Base() {
cout<<"this is subclass Base"<};
/* instance class and sample */
void main()
{
Virtualbase* inst = new SubVirtual(); //multstate pointer
inst->Demon();
inst->Base();
// inst = new Virtualbase();
// inst->Base()
return ;
}

VXWORKS Kernel Analysis

Leo Liu — Fri, 05 Jun 2026 07:26:22 +0000

Originally published on the Sienovo Engineering Blog. Sienovo is the overseas brand of 深圳信迈 (Shenzhen Xinmai), building edge AI computing solutions for industrial video analytics.

VXWORKS Kernel Analysis

Structure of Real-time Operating Systems

The most primitive structural form of operating systems developed in the early days of computing was a monolithic entity. In such systems, modules providing different functionalities, such as processor management, memory management, and input/output, were typically independent. However, they did not consider other modules in use during their execution, and each module ran with the same time granularity.

As modern real-time environments require many different functionalities, and due to the asynchrony and non-determinism caused by concurrent activities in such environments, operating systems have become more complex. Therefore, the monolithic structure of early operating systems has been superseded by more precise internal structures.

The Starting Point of Hierarchical Structure - The Kernel

The best internal structural model for an operating system is a hierarchical one, with the kernel at the lowest layer. These layers can be seen as an inverted pyramid, with each layer building upon the functionalities of the lower layers. The kernel contains only the most important low-level functions executed by an operating system. Like a monolithic operating system, the kernel provides an abstraction layer between high-level software and low-level hardware. However, the kernel only provides a minimal set of operations required to construct the other parts of the operating system.

Requirements for a Real-time Kernel

A real-time operating system kernel must satisfy several fundamental requirements imposed by specific real-time environments. These include:

Multitasking: Due to the asynchronous nature of real-world events, it is crucial to be able to run many concurrent processes or tasks. Multitasking provides a better match to the real world because it allows multithreaded execution corresponding to many external events. The system kernel allocates the CPU to these tasks to achieve concurrency.

Preemptive Scheduling: Real-world events have inherent priorities, and these priorities must be considered when allocating the CPU. With priority-based preemptive scheduling, tasks are assigned priorities, and among the tasks that are ready to execute (not suspended or waiting for resources), the highest-priority task is allocated CPU resources. In other words, when a higher-priority task becomes executable, it immediately preempts the currently running lower-priority task.

Fast and Flexible Inter-task Communication and Synchronization: In a real-time system, many tasks may execute as part of an application. The system must provide fast and powerful communication mechanisms between these tasks. The kernel must also provide synchronization mechanisms required for efficiently sharing non-preemptable resources or critical sections.

Convenient Communication Between Tasks and Interrupts: Although real-world events often arrive as interrupts, to provide effective queuing, prioritization, and reduced interrupt latency, it is often desirable to handle the corresponding work at the task level. Therefore, communication between the task level and the interrupt level is necessary.

Performance Bounds: A real-time kernel must provide worst-case performance optimization, rather than throughput optimization. We prefer a system that can consistently execute a function in 50 microseconds over one that executes it in 10 microseconds on average but occasionally takes 75 microseconds.

Special Considerations: As the requirements for real-time kernels increase, the need for the kernel to support increasingly complex functionalities must be considered. This includes multiprocessing, Ada, and support for newer, more powerful processor architectures like RISC.

Kernels with Other Names

Many commercial kernels support far more functionality than the requirements listed above. In this respect, they are not true kernels but more like small monolithic operating systems, as they include simple memory allocation, clock management, and even some input/output system calls.

This classification is not merely a semantic argument; later sections of this article will illustrate the importance of limiting kernel functionality and optimizing these functions.
VxWorks Kernel: Wind

The VxWorks operating system is one of the most fully-featured, processor-independent real-time systems available today. However, VxWorks is a hierarchical structure with a relatively small, true microkernel. The kernel only provides a multitasking environment, inter-process communication, and synchronization functions. These functional modules are sufficient to support the rich performance requirements provided by VxWorks at higher levels. Kernel operations are typically invisible to the user. Applications use system calls to implement task management and synchronization that require kernel involvement, but the processing of these calls is invisible to the calling task. Applications simply link the appropriate VxWorks routines (often using VxWorks' dynamic linking capability) and issue system calls as if calling subroutines. This interface is not like some systems that require a clumsy jump table interface where the user specifies a kernel function call via an integer.

Multitasking

The fundamental function of the kernel is to provide a multitasking environment. Multitasking makes many programs appear to execute concurrently, while in fact, the kernel executes them in segments according to basic scheduling algorithms. Each seemingly independent program is called a task. Each task has its own context, which includes the CPU environment and system resources it sees when the kernel schedules it for execution.

Task States

The kernel maintains the current state of each task in the system. State transitions occur when an application calls a kernel function service. The Wind kernel states are defined as follows:

Ready state – A task is not waiting for any resources other than the CPU.
Blocked state – A task is blocked because some resource is unavailable.
Delayed state – A task is sleeping for a specified period.
Suspended state – An auxiliary state primarily used for debugging; suspension prevents task execution.

A task enters the suspended state after creation. Specific operations are required to move the created task into the ready state. This operation is very fast, allowing applications to create tasks in advance and activate them quickly.

Scheduling Control

Multitasking requires a scheduling algorithm to allocate the CPU to ready tasks. The default scheduling algorithm in VxWorks is priority-based preemptive scheduling, but applications can also choose to use round-robin scheduling.

Priority-based Preemptive Scheduling: With priority-based preemptive scheduling, each task is assigned a priority, and the kernel allocates the CPU to the highest-priority task in the ready state. Scheduling is preemptive because when a task with a higher priority than the current task becomes ready, the kernel immediately saves the current task's context and switches to the higher-priority task's context. VxWorks has 256 priorities, from 0 to 255. Tasks are assigned a priority at creation, and this priority can be dynamically modified during task execution to track real-world event priorities. External interrupts are assigned a priority higher than any task, allowing them to preempt a task at any time.

Round-Robin: Priority-based preemptive scheduling can be extended with round-robin scheduling. Round-robin scheduling allows ready tasks of the same priority to share the CPU fairly. Without round-robin scheduling, when multiple tasks share the processor at the same priority, one task might monopolize the CPU, not being blocked until preempted by a higher-priority task, and not giving other tasks of the same priority a chance to run. If round-robin is enabled, the execution time counter for the running task increments with each clock tick. When the specified time slice expires, the counter is reset, and the task is placed at the end of the queue for tasks of the same priority. New tasks joining a specific priority group are placed at the end of that group's queue, and their run counter is initialized to zero.

Basic Task Functions

Basic task functions for state control include task creation, deletion, suspension, and resumption. A task can also put itself to sleep for a specific interval.
Many other task routines provide state information obtained from the task context. These routines include access to a task's current processor register control.

Task Deletion Issues

The Wind kernel provides mechanisms to prevent tasks from being accidentally deleted. Typically, a task executing in a critical section or accessing a critical resource needs special protection. Consider the following scenario: a task obtains exclusive access to some data structure, and while it is executing within the critical section, it is deleted by another task. Since the task cannot complete its operation on the critical section, the data structure may be left in a corrupted or inconsistent state. Moreover, if the task has no opportunity to release the resource, then no other task can now acquire that resource; the resource is frozen.

Any task attempting to delete or terminate a task that has set deletion protection will be blocked. When the protected task completes its critical section operation, it will cancel deletion protection to allow itself to be deleted, thereby unblocking the deleting task.

As shown above, task deletion protection is often accompanied by mutual exclusion operations.
Thus, for convenience and efficiency, mutex semaphores include a deletion protection option. (See "Mutex Semaphores")

Inter-task Communication

To provide full multitasking system functionality, the Wind kernel offers a rich set of inter-task communication and synchronization mechanisms. These communication functions enable individual tasks within an application to coordinate their activities.

Shared Address Space

The basis of the Wind kernel's inter-task communication mechanism is the shared address space where all tasks reside. Through a shared address space, tasks can communicate freely using pointers to shared data structures. Pipes do not require mapping a memory region into the address spaces of two communicating tasks.

Unfortunately, while shared address space offers the aforementioned advantages, it introduces the risk of unprotected reentrant access to memory. UNIX operating systems provide such protection by isolating processes, but this comes with a significant performance penalty for real-time operating systems.

Mutual Exclusion Operations

While a shared address space simplifies data exchange, it becomes necessary to avoid resource contention through mutually exclusive access. Many mechanisms used to achieve mutually exclusive access to a resource differ only in the scope of their mutual exclusion. Methods for achieving mutual exclusion include interrupt disabling, task preemption disabling, and resource locking via semaphores.

Interrupt Disabling: The strongest mutual exclusion method is masking interrupts. Such a lock guarantees exclusive access to the CPU. While this method can certainly solve the mutual exclusion problem, it is inappropriate for real-time systems because it prevents the system from responding to external events during the lock. Long interrupt latencies are unacceptable for applications requiring deterministic response times.

Preemption Disabling: Disabling preemption provides a weaker form of mutual exclusion. Other tasks are not allowed to preempt the current task while it is running, but interrupt service routines (ISRs) can execute. This can also lead to poor real-time responsiveness, similar to interrupt disabling, as blocked tasks may experience significant preemption latency, and ready high-priority tasks may be forced to wait an unacceptable amount of time before they can execute. To avoid this, semaphores should be used for mutual exclusion whenever possible.

Mutex Semaphores: Semaphores are the fundamental way to lock access to shared resources. Unlike disabling interrupts or preemption, semaphores limit mutual exclusion operations to only the relevant resources. A semaphore is created to protect a resource. VxWorks semaphores follow Dijkstra's P() and V() operation model.

When a task requests a semaphore, P(), two things can happen, depending on the set or cleared state of the semaphore at the time of the call. If the semaphore is in the set state, it is cleared, and the task immediately continues execution. If the semaphore is in the cleared state, the task is blocked to wait for the semaphore.

When a task releases a semaphore, V(), several things can happen. If the semaphore is already in the set state, releasing it has no effect. If the semaphore is in the cleared state and no tasks are waiting for it, the semaphore is simply set. If the semaphore is in the cleared state and one or more tasks are waiting for it, the highest-priority task is unblocked, and the semaphore remains in the cleared state.

By associating resources with semaphores, mutual exclusion operations can be achieved. When a task wants to operate on a resource, it must first acquire the semaphore. As long as the task owns the semaphore, all other tasks requesting that semaphore are blocked. When a task finishes using the resource, it releases the semaphore, allowing another task waiting for the semaphore to access the resource.

The Wind kernel provides binary semaphores to address problems caused by mutual exclusion operations. These problems include deletion protection for resource owners and priority inversion caused by resource contention.

Deletion Protection – One problem caused by mutual exclusion involves task deletion. In a critical section protected by a semaphore, it is necessary to prevent the executing task from being accidentally deleted. Deleting a task executing in a critical section is catastrophic. The resource will be corrupted, and the semaphore protecting the resource will become unavailable, making the resource inaccessible. Deletion protection is usually provided in conjunction with mutual exclusion operations. For this reason, mutex semaphores often provide options to implicitly offer the task deletion protection mechanisms mentioned earlier.

Priority Inversion / Priority Inheritance – Priority inversion occurs when a high-priority task is forced to wait for an indefinite period for a lower-priority task to complete execution. Consider the following hypothesis:

T1, T2, and T3 are high, medium, and low priority tasks, respectively. T3 acquires a related resource by owning a semaphore. When T1 preempts T3 and requests the same

Detailed Explanation of ANSI C Standard File I/O Functions

Leo Liu — Thu, 04 Jun 2026 07:34:43 +0000

Originally published on the Sienovo Engineering Blog. Sienovo is the overseas brand of 深圳信迈 (Shenzhen Xinmai), building edge AI computing solutions for industrial video analytics.

I. Overview

The ANSI C file system is built upon the buffered file system of earlier C versions (also known as formatted or high-level file systems).
Difference between Streams and Files: The level of abstraction provided by C's I/O system between the programmer and the device being used is called a stream, while the physical device is called a file. The C file system can operate on various devices such as terminals, disk drives, and tape drives. Regardless of how different these devices are, the ANSI file system converts them into logical devices called "streams," providing great device independence. In C, the logical concept of a file refers to anything from a disk file to a terminal printer. A stream is associated with a file by performing an open operation. Once a file is opened, information can be exchanged between the program and that file. Not all files have the same capabilities; for example, disk files support random access, but terminals do not. This illustrates an important characteristic of the C I/O system: all streams are similar, but files are different.
There are two types of streams: text streams and binary streams. A text stream is a sequence of characters. In a text stream, specific character conversions are required by the host environment, so there is no one-to-one correspondence between the characters written (or read) and the characters stored in the device. Similarly, due to possible conversions, the number of characters written (or read) may not match the number of characters stored in the device. A binary stream, on the other hand, is a sequence of bytes where the storage in the device is one-to-one, meaning there are no character conversions. Therefore, the number of bytes written (or read) matches the number of bytes stored in the device.
Files are linked to streams through a close operation. For output streams, when a file is closed, the associated stream is written to the device, a process commonly referred to as flushing. This ensures that no information is left in the disk buffer. When a program ends, all files are automatically closed; however, if a program crashes, files may not be closed properly, meaning information might not be written to disk. Each stream associated with a file has a control structure of type FILE, which is defined in stdio.h, and cannot be modified directly.
The following functions and types are defined in stdio.h:

- `fopen()`
- `fclose()`
- `putc()`
- `fputc()`
- `getc()`
- `fgetc()`
- `fseek()`
- `fprintf()`
- `fscanf()`
- `feof()` — returns true if the end of the file is reached
- `ferror()`
- `rewind()`
- `remove()` — deletes a file
- `fflush()` — flushes a file
- `fread()` — reads from a file
- `fwrite()` — writes to a file

Additionally, types such as `size_t` (large enough to hold the result of subtracting two pointers, a variant of `unsigned int`), `fpos_t` (used to describe a specific position in a file, also a variant of `unsigned int`), and the `FILE` type are defined. Several macros are also defined: `EOF` (typically defined as -1), `SEEK_SET`, `SEEK_CUR`, and `SEEK_END`.

When operating on files in read/write mode, two points must be noted: First, if a write operation is followed by a read operation, the fflush() function or file positioning functions such as fseek() or rewind() must be called. Second, if a read operation is followed by a write operation, it must be at the end of the file or a file positioning function must be called between the two operations.

II. Function Details

Opening and Closing Files

(1) Opening Files: The function to open files in the ANSI C library is declared as follows:

FILE *fopen(const char *restrict filename, const char *restrict modes);

If successful, it returns a pointer to the opened file. If it fails, it returns NULL.

The first parameter is a pointer to the string of the filename to be opened (e.g., /etc/service). The second parameter specifies the mode for opening the file. The opening modes are as follows:

Parameter	Description
`r` (or `rb`)	Opens the file in read-only mode; the file must exist.
`r+` (or `rb+`)	Opens the file in read/write mode; the file must exist.
`w` (or `wb`)	Opens the file in write-only mode; if the file exists, it is cleared; if it does not exist, it is created.
`w+` (or `wb+`)	Opens the file in read/write mode; if the file exists, it is cleared; if it does not exist, it is created.
`a` (or `ab`)	Opens the file in write-only mode for appending; if the file exists, data is appended to the end; if it does not exist, it is created.
`a+` (or `ab+`)	Opens the file in read/write mode for appending; if the file exists, data can be read/written at the end; if it does not exist, it is created.

(2) Closing Files: The function to close a file is declared as:

int fclose(FILE *stream);

To close all opened stream objects, the fcloseall function can be used:

int fcloseall(void);

To update the contents of the buffer, even if the buffer is not full, the fflush function can be used:

int fflush(FILE *stream);

Reading and Writing File Streams

(1) Character Read/Write Operations:

(a) Character Read Operation: This operation reads one character at a time from the stream. The related function declarations are:
```
int fgetc(FILE *stream); // Reads one character from the stream
int getchar(void); // Reads one character from standard input
int getc(FILE *stream); // Equivalent to fgetc
```
(b) Character Write Operation: This operation writes one character at a time to the stream. The relevant function declarations are:
```
int fputc(int c, FILE *stream); // Writes a character to an output file stream
int putc(int c, FILE *stream); // Similar to fputc
int putchar(int c); // Equivalent to putc(c, stdout)
```
(2) Line Read/Write Operations:

(a) Line Read Operation: This operation reads one line of characters from the stream. The fgets function reads a string from the input file stream and writes it to the string pointed to by s, stopping at a newline or end-of-file marker, and appends a null byte (\0) at the end.
```
char *fgets(char *s, int n, FILE *stream);
```
The gets function is similar to fgets, but it reads from standard input and discards the newline character.

(b) Line Write Operation: This operation writes one line of characters to the standard output. The puts function writes the string pointed to by s (terminated with a null character) to standard output, followed by a newline. The fputs function writes the string pointed to by s to the specified output stream but does not append a newline.
```
int fputs(char *s, FILE *stream);
int puts(char *s);
```
(3) Block Read/Write Operations:

(a) Block Read Operation:
```
int fread(void *buffer, int size, int count, FILE *fp);
```
fread() reads size bytes from the file pointed to by fp starting from the current position, repeating this count times, and stores the data in the memory starting at buffer.

(b) Block Write Operation:
```
int fwrite(void *buffer, int size, int count, FILE *fp);
```
fwrite() outputs size bytes from buffer to the file pointed to by fp, repeating this count times. This is generally used for handling binary files.

(c) Example of Writing a String to a File:
```
char *str = "hello, I am a test program!";
fwrite(str, sizeof(char), strlen(str), fp);
```
To write a character array to a file:
```
char str[] = {'a', 'b', 'c', 'd', 'e'};
fwrite(str, sizeof(char), sizeof(str), fp);
```
To write an array of integers to a file:
```
int a[] = {12, 33, 23, 24, 12};
size_t nmemb = sizeof(a) / sizeof(a[0]);
fwrite(a, sizeof(int), nmemb, fp);
```
Note: Since the generated file is a binary file rather than a text file, the representation of integers may vary across machines, so it cannot be opened directly. You can use the fread function to verify if the data has been written to the file.
File Stream Positioning

(1) Returning Current Read/Write Position:
```
long int ftell(FILE *stream);
```
If successful, it returns the current position of the pointer in bytes from the beginning of the file; if it fails, it returns -1.

(2) Modifying Current Read/Write Position:

The usage of fseek is as follows:
```
int fseek(FILE *stream, long offset, int fromwhere);
```
The first parameter is the file pointer, the second parameter is the offset to move, and the third parameter indicates where to move from, using three macros:

- `SEEK_SET` (0) — beginning of the file
- `SEEK_CUR` (1) — current position in the file
- `SEEK_END` (2) — end of the file

It is recommended to use macros instead of numbers. In summary:

```c
fseek(fp, 100L, SEEK_SET); // Move the pointer 100 bytes from the start of the file
fseek(fp, 100L, SEEK_CUR); // Move the pointer 100 bytes from the current position
fseek(fp, 100L, SEEK_END); // Move the pointer back 100 bytes from the end of the file
```

This function is commonly used to calculate the length of a stream:

```c
int filesize = fseek(fp, 0, SEEK_END);
fseek(fp, 0, SEEK_SET);
```

(3) **Resetting Current Read/Write Position:** After completing one operation, to prepare for the next operation, the `rewind` function should be called to reset the read/write position to the beginning of the file.

```c
void rewind(FILE *stream);
```

File Stream Error Detection: To identify an error, many stdio library functions return an out-of-bounds value, such as a null pointer or the constant EOF. In these cases, the errors are indicated by the external variable errno:
```
#include <errno.h>
extern int errno;
```
It is important to note that many functions may change the value of errno. The value is only valid when a function call fails. It is advisable to check the value of errno immediately after a function indicates failure. Before using it, copy its value to another variable, as some printing functions, like fprintf, may modify its value.

You can also determine whether an error has occurred or if the end of the file has been reached by checking the status of the file stream.

To check if the end of the file has been reached:
```
int feof(FILE *stream);
```
If the end of the file is reached, it returns 1; otherwise, it returns 0.

To check for errors in a given stream:
```
int ferror(FILE *stream);
```
If no error has occurred, it returns 0; otherwise, it returns an error, which is stored in errno. When using the above two functions for file stream detection, the error flag will be set. After error handling, the error flag should be cleared:
```
void clearerr(FILE *stream);
```
The clearerr function clears the end-of-file or error indicator for the file stream pointed to by stream. This function does not return a value or define an error. You can use this function to recover from error conditions on the stream, such as when the disk is full and data needs to be rewritten to the file stream.
Streams and File Descriptors: Each file stream corresponds to a lower-level file descriptor. While it is possible to mix low-level input/output with high-level file stream operations, it is generally unwise due to unpredictable buffer effects.
```
#include <stdio.h>
int fileno(FILE *stream); // Returns the file descriptor for a given file stream
FILE *fdopen(int fildes, const char *mode); // Creates a new file stream based on an existing file descriptor
```
By calling the fileno function, you can determine which lower-level file descriptor is being used by a file stream. It returns a file descriptor for the specified file stream; if it fails, it returns -1. If you need low-level access to an open stream, you can use this function, such as with fstat.

The fdopen function allows you to create a new file stream based on an already opened file descriptor. Essentially, this function provides a stdio buffer for an already opened file descriptor, which can be useful for clarity. The fdopen function operates similarly to fopen, except that it uses a lower-level file descriptor. If you want to use open to create a file, perhaps for better permission control, but wish to perform write operations using file streams, this function becomes particularly useful. The mode parameter is the same as that for fopen, and it must be compatible with the file access mode established when the file was initially opened. fdopen returns a new file stream; if it fails, it returns NULL.
File and Directory Maintenance: The standard library and system calls provide complete control over file creation and maintenance.

- **`chmod`:** You can use the `chmod` system call to change the permissions of a file or directory. The syntax is as follows:

```c
#include <sys/stat.h>
int chmod(const char *path, mode_t mode);
```

The file specified by `path` will have the permissions specified by `mode`. The `mode` specified is similar to that in the `open` system call, being a bitwise OR of the desired permissions. Unless the program is granted appropriate permissions, only the file's owner or a superuser can change its permissions.

- **`chown`:** A superuser can use the `chown` system call to change the owner of a file. The syntax is as follows:

```c
#include <unistd.h>
int chown(const char *path, uid_t owner, gid_t group);
```

This call uses the numeric user ID or group ID (which can be obtained via `getuid` and `getgid`) and a constant to determine who can change the file's owner. With appropriate permissions set, we can change both the owner and the group of a file.

- **`unlink`, `link`, `symlink`:** You can use `unlink` to remove a file. `unlink` removes the directory entry for a file and decreases its link count. If the function call is successful, it returns 0; if it fails, it returns -1. You must have write and execute permissions in the directory where the command is executed, as files have their own directory entries.

The syntax is as follows:

```c
int unlink(const char *path);
int link(const char *path1, const char *path2);
int symlink(const char *path1, const char *path2);
```

If the link count reaches 0 and no processes are using the file, the file will be deleted. In fact, a directory entry is always deleted, but the file's space will not be reclaimed until the last associated process is closed. The `rm` program uses this call. Typically, we can use the `ln` program to create a link for a file. We can use the `link` system call to create a planned link for a file. The `link` system call creates a new link for an existing file specified by `path1`, with the new directory entry specified by `path2`. Similarly, we can use `symlink` to create a symbolic link. It is important to note that a file's symbolic link will not prevent the file from being deleted, unlike a hard link.

Power Data Acquisition and Transmission Design Based on OMAPL138 + Xilinx Spartan-6

Leo Liu — Wed, 03 Jun 2026 07:37:59 +0000

Originally published on the Sienovo Engineering Blog. Sienovo is the overseas brand of 深圳信迈 (Shenzhen Xinmai), building edge AI computing solutions for industrial video analytics.

Ma Shixi and Yang Han, researchers from Wuhan Fiberhome Furi Electric Co., Ltd., presented in the 10th issue of "Electrical Engineering" magazine in 2015 a power data acquisition and transmission design based on OMAPL138. This solution is built upon Sienovo's OMAPL138+SPARTAN6 development board, with the OMAPL138 microprocessor at its core. The microprocessor analyzes and processes acquired data such as voltage, current, and digital inputs, then transmits the processed data to upper-level host systems via interfaces like RS485/RS232 and Ethernet using various communication protocols. The article details the functions of each module, hardware design, and software design flow. This design is flexible, convenient, and offers significant practical value.

1 Evaluation Board Overview

Based on TI OMAP-L138 (fixed/floating-point DSP C674x + ARM9) + Xilinx Spartan-6 FPGA processor;
OMAP-L138 and FPGA are connected via uPP, EMIFA, and I2C buses, supporting communication speeds up to 228 MByte/s; OMAP-L138 operates at a main frequency of 456 MHz, delivering up to 3648 MIPS and 2746 MFLOPS computing performance;
FPGA compatible with Xilinx Spartan-6 XC6SLX9/16/25/45, offering strong platform scalability;
The development board exposes rich peripherals, including Gigabit Ethernet, SATA, EMIFA, uPP, USB 2.0, and other high-speed data transmission interfaces, as well as common interfaces such as GPIO, I2C, RS232, PWM, and McBSP;
Passed high and low-temperature testing certification, suitable for harsh operating environments;
Triple-core (DSP+ARM+FPGA) core board measuring 66mm × 38.6mm, using industrial-grade B2B connectors to ensure signal integrity;
Supports bare-metal, SYS/BIOS, and Linux operating systems.

In power system operations, large volumes of data from electrical equipment—such as three-phase voltages, three-phase currents, and switch status signals—must be collected and transmitted over long distances. As power systems continue to evolve, increasingly higher demands are placed on the accuracy of data acquisition and the timeliness of data transmission[1].

Traditional low-end measurement instruments based on microcontrollers suffer from poor data processing capabilities and limited module functionality. Mid-to-high-end instruments based on DSPs take advantage of DSPs' strong data processing performance but often lack sufficient storage capacity and communication capabilities.

Since the International Electrotechnical Commission (IEC) adopted IEC61850 as the standard protocol for seamless communication in power systems in 2000, IEC61850 has gradually been applied in power systems and substation automation both domestically and internationally[2]. However, many existing meters and automated monitoring devices still fail to meet IEC61850 requirements. In practice, both Modbus and IEC61850 protocols often coexist, and devices supporting only a single protocol face significant limitations in application.

To address this issue, this paper proposes a data acquisition and transmission device design based on OMAPL138 that supports multiple communication protocols. The proposed solution can simultaneously support two different protocols, offering flexibility, convenience, and high practical value.

1 Overall Design

The hardware design is based on the OMAPL138 processor, a dual-core embedded microprocessor combining ARM and DSP, developed by Texas Instruments. It features rich peripheral interfaces, including DDR2 controller, Ethernet controller, UARTs, SPI, I2C, LCD controller, USB, and others[3].

The FPGA is compatible with Xilinx Spartan-6 XC6SLX9/16/25/45, enabling strong platform upgrade capability. The use of an OMAPL138+SPARTAN6 core board facilitates secondary development.

In this design, the OMAPL138 microprocessor connects to Nand-Flash via the EMIFA bus. The Nand-Flash stores UBOOT, the operating system kernel, file system, application programs, and related data. OMAPL138 connects to a DDR2SDRAM chip via the DDR2 bus, which is used for running the operating system and applications and storing temporary files.

As shown in Figure 1, OMAPL138 connects to an external analog-to-digital converter (ADC) via the SPI interface to acquire analog signals. It uses universal asynchronous serial ports (UARTs) to multiplex between RS232 and RS485 modes, transmitting acquired data using the Modbus protocol from the acquisition terminal to the backend system.

OMAPL138 connects its internal Ethernet controller to an external Ethernet PHY chip via the MII interface, enabling Ethernet communication. Collected data is sent to the backend using the IEC61850 protocol over the Ethernet interface.

Figure 1 Hardware System Block Diagram

2 Hardware Design

2.1 Acquisition Module Design

1) Analog Input Acquisition Module

The design uses ADI's AD7689 as the analog-to-digital converter, which features 8 analog input channels, 16-bit resolution, and a sampling rate of 250 kSPS[4]. The AD7689 converts analog signals into digital form and sends the digitized data to the processor via the SPI bus for further processing.

Configuration in this design is as follows: (1) The ADC is powered by 5V, with a reference voltage of 4.096V provided by an internal voltage reference; (2) The ADC's digital output voltage is 3.3V, matching the processor's IO voltage level; (3) The analog input signals are unipolar, converted from differential signals from current/voltage transformers via a signal conditioning circuit; (4) The data output rate is controlled by the processor.

2) Digital Input Acquisition Module

Digital input signals (switch status) are optically isolated and then directly fed into the IO pins of the OMAPL138 processor. Software configures these IO pins as inputs to capture external digital signals, which are then stored in shared memory and subsequently transmitted using the appropriate communication protocol.

2.2 Communication Module Design

The design supports both Modbus and IEC61850 protocols, enabling flexible adaptation to different communication requirements. Modbus can be implemented via either RS232 or RS485, with software-configurable multiplexing and automatic switching between the two. IEC61850 communication uses either optical or electrical Ethernet interfaces.

1) RS232/RS485 Multiplexed Interface

The circuit diagram for the RS232/RS485 multiplexing scheme is shown in Figure 2. The OMAPL138 UART's transmit pin connects to both the transmit input of the RS232 module (MAX3232) and the transmit input of the RS485 module (MAX485). Similarly, the UART's receive pin connects to both the receive output of MAX3232 and MAX485.

Two IO pins of OMAPL138 drive the set and reset coils of a dual-coil latching miniature relay. Pulse signals on these IO pins switch the relay contacts between set and reset states.

To enhance the driving capability of the OMAPL138 IO pins, this design uses the IO pins to drive transistors, which in turn drive the relays. This allows the processor to control which interface—RS232 or RS485—is connected to the external communication lines by changing the state of its IO pins. Thus, the two communication methods can be easily multiplexed through software configuration.

Figure 2 RS232/RS485 Multiplexed Communication Module

2) Ethernet Communication Interface Design

The OMAPL138 processor includes an Ethernet controller supporting 10/100 Mbps Ethernet. By connecting its MII or RMII interface to an external Ethernet PHY chip, Ethernet communication is achieved. This design uses Micrel's KSZ8001 as the Ethernet PHY chip, supporting both optical and electrical 10/100 Mbps Ethernet. The circuit diagram for the OMAPL138 and KSZ8001 Ethernet interface is shown in Figure 3.

Figure 3 Ethernet Communication Interface Circuit Diagram

2.3 Power Supply Design

The design requires three power supply voltages: 1.2V, 1.8V, and 3.3V, which are used for the processor core, DDR2 controller, processor IO, and other chip operations, respectively. These voltages are generated from an external 5V supply using dedicated power management ICs: Intersil's ISL6410A and ISL65426. ISL6410A provides 1.2V output, while ISL65426 provides 1.8V and 3.3V outputs.

ISL6410A and ISL65426 are high-performance PWM-modulated DC/DC converters from Intersil, capable of delivering stable voltage and current output across an industrial temperature range of -40°C to 85°C with high conversion efficiency[5][6].

Both chips integrate internal MOSFETs and include properly designed power-good and enable pins, greatly simplifying user design.

When designing the power supply with these ICs, the following points should be noted:

(1) Since the OMAPL138 processor has specific power-up sequencing requirements, it is essential that 1.2V is established before 1.8V, and 1.8V before 3.3V. The enable pin (EN) of ISL6410A is pulled up to 5V. Its power-good pin (PG) is connected to the enable pin (EN1) of the 1.8V output channel of ISL65426. The power-good pin (PG1) of the 1.8V output channel is connected to the enable pin (EN2) of the 3.3V output channel. This ensures proper power-up sequencing.

(2) The output configuration pin (VSET) of ISL6410A is pulled up to 5V, ensuring that ISL6410A correctly outputs 1.2V when supplied with 5V input.

(3) ISL65426 has two output channels and six adjustable power output modules, allowing independent configuration. Output voltages can be set via programming pins; for non-standard voltages, feedback via resistive dividers can be used. Since 1.8V and 3.3V are supported by programming pins in this design, pins V1SET1, V1SET2, V2SET1, and V2SET2 are all pulled up to 5V. Given that the 3.3V supply draws more current than the 1.8V supply in this design, the current configuration pins ISET1 and ISET2 are set to 1 and 0, respectively.

3 Software Design

The software consists of five components: ubl, uboot, Linux kernel, file system, and application programs. The ubl and uboot handle initialization of OMAPL138 and loading/launching the kernel.

The Linux kernel version used is 2.6.32, extended with custom serial port drivers. The application layer supports multiple communication protocols, including ModbusRTU and IEC61850 for data reporting. After processing the acquired signals using algorithms, the CPU writes the results into shared memory for use by various protocol modules in packet encapsulation.

3.1 Low-Level Serial Port Driver

The Modbus protocol is supported via serial communication with upper-level devices. Serial data transmission and reception are handled by kernel-level serial drivers. The Linux serial driver architecture comprises three layers: low-level driver, line discipline, and tty device driver core, as illustrated in Figure 4.

As shown in Figure 4, user-space commands and data are passed to the low-level driver, processed and scheduled by the line discipline, and then passed to the tty core layer, which performs hardware operations according to the command. In some cases, data may bypass the line discipline and go directly from the low-level driver to the core driver.

3.2 Data Acquisition and Processing

1) Data Accuracy Algorithm

To minimize anomalies in A/D acquisition data caused by environmental interference or hardware issues, the design applies algorithmic processing before storing data in shared memory. The algorithm discards the highest and lowest values from ten consecutive samples and computes the average of the remaining eight values as the valid signal for that acquisition cycle. While this reduces data acquisition efficiency, it improves data reliability and is suitable for applications where high sampling rates are not critical.

Figure 4 Serial Port Driver Architecture

2) Data Transfer

To ensure efficient data transfer between protocols, this design uses shared memory. The data structures for different protocols map to the same physical memory block. During main process startup, a memory block is allocated. The configuration file is then read to define the mapping of data structure members across protocols, enabling seamless inter-protocol data sharing.

3.3 Communication Protocols

The design supports the ModbusRTU protocol, acting as a slave device that responds to various operations initiated by upper-level devices via ModbusRTU. Communication with upper-level devices occurs via either RS485 or RS232, with the interface type selectable via configuration file. Considering the half-duplex nature of RS485, appropriate delays are introduced between sending and receiving to ensure data integrity.

The system also supports the IEC61850 protocol, transmitting data to the backend via Ethernet. Data for both protocols originates from shared memory. The system module block diagram is shown in Figure 5. As seen in Figure 5, both protocol modules operate as independent task processes, allowing the system to simultaneously support both protocols and serve multiple types of upper-level devices.

3.4 Data Logging

The design includes comprehensive fault logging, saving abnormal operational data and fault timestamps to log files stored in flash memory for easy troubleshooting by maintenance personnel. Additionally, historical data logging is supported, recording each data change along with its timestamp into log files stored in flash. The system supports up to 128MB of log file storage.

Figure 5 Data Acquisition and Shared Memory Module Diagram

4 Testing and Verification

To verify the data acquisition functionality and the Modbus and IEC61850 communication capabilities, the following test setup was used: A relay protection tester outputs three-phase current, three-phase voltage, and digital signals to the device under test. The device acquires and processes the data, then transmits it to a PC using both Modbus and IEC61850 protocols.

For Modbus communication verification, the PC sends Modbus polling messages to the device via a serial debugging tool and checks the response data.

IEC61850 communication is verified using the Ethernet packet capture tool MMS Ethereal to capture MMS messages and inspect data transmission. Screenshots of the transmitted protocol messages are shown in Figures 6 and 7. Comparison of the protocol messages with the data output from the relay protection tester confirms that the design meets the original requirements.

Figure 6 Modbus Polling and Response

Figure 7 MMS Message

5 Conclusion

This paper presents a power data acquisition and multi-protocol transmission solution based on OMAPL138. The modular design supports multiple communication protocols for data transmission, greatly enhancing user convenience. It also provides a robust hardware and software platform for future expansion of additional functionalities and protocols, demonstrating high practical value.

ZYNQ Lidar-Visual SLAM Algorithm Porting and Design

Leo Liu — Tue, 02 Jun 2026 07:35:04 +0000

Originally published on the Sienovo Engineering Blog. Sienovo is the overseas brand of 深圳信迈 (Shenzhen Xinmai), building edge AI computing solutions for industrial video analytics.

This article details the development and implementation of a ZYNQ-based system for real-time 3D reconstruction, leveraging both FPGA-accelerated stereo vision and a LiDAR-based mobile platform. Readers will learn about the system's architecture, key algorithms like BM, SGBM, and ICP, hardware components including a Mecanum wheel robot and R-Fans-16 LiDAR, and its performance characteristics for environmental perception in applications like robotics and autonomous systems.

FPGA-Accelerated Stereo Vision on ZYNQ

The initial phase of this project focused on developing an FPGA-accelerated stereo vision system on a Xilinx ZYNQ platform for real-time 3D reconstruction. This involved a tight integration of the ZYNQ's Processing System (PS) and Programmable Logic (PL) for collaborative processing, enabling high-performance image processing tasks.

Key achievements in this stereo vision component include:

Stereo Image Acquisition and PS+PL Processing: Successfully implemented the capture of stereo images, utilizing the ZYNQ's PS for control and higher-level tasks, and the PL (FPGA fabric) for accelerating computationally intensive image processing operations.
Camera Calibration and Lens Correction: Developed and integrated routines for precise camera calibration and lens distortion correction directly on the ZYNQ platform. This is crucial for accurate 3D reconstruction, as it eliminates geometric distortions introduced by the camera lenses.
Stereo Matching Algorithms: Implemented and optimized several stereo matching algorithms on ZYNQ. This currently includes both local stereo matching (Block Matching - BM) and semi-global stereo matching (Semi-Global Block Matching - SGBM), allowing for the computation of disparity maps from stereo image pairs.
Depth Map and Rainbow Map Generation: After stereo matching, the system converts the generated disparity maps into real-time depth maps. These depth maps are further processed into "rainbow maps," which are colorized depth visualizations that provide an intuitive representation of scene depth.
Ranging Algorithm: A ranging algorithm has been developed on ZYNQ to calculate distances. However, it currently lacks a recognition algorithm, meaning it can only compute the distance to a single specified point in the image. Despite this limitation, the system achieves an accuracy within 10mm, with an effective testing range of up to 5m.
Algorithm Optimization: All implemented algorithms have undergone significant optimization. The system currently achieves near 720p image quality at a frame rate of approximately 30 frames per second, demonstrating its capability for real-time applications.

A dedicated series of articles is planned to detail the implementation process of this real-time 3D reconstruction project on ZYNQ, covering its development from scratch.

The Broader Context of 3D Reconstruction

In recent years, fields such as machine learning and autonomous driving have become prominent research areas, where environmental perception by computers is paramount. Consequently, 3D environment reconstruction is an essential trend, facilitating interaction between virtual and real worlds.

The two primary methods for 3D reconstruction are vision-based and LiDAR-based technologies. Vision-based ranging relies on triangulation, typically offering a maximum range of 5-8m, making it less suitable for larger spaces. It is also highly susceptible to lighting conditions. In contrast, LiDAR (Light Detection and Ranging) can be applied in a much wider array of scenarios due offering superior range and robustness to ambient light.

Sienovo's ZYNQ-based LiDAR 3D Modeling System

This article's primary focus is on a ZYNQ-based LiDAR 3D modeling system designed to address the challenges of environmental perception. This system aims to deeply explore target environments, collect point cloud data, reconstruct 3D spatial models, and support various machine vision applications such as measurement.

Application Areas

3D modeling has extensive applications across various domains:

Drones and Autonomous Vehicles: Essential for real-time obstacle avoidance and path planning.
Machine Vision and 3D Printing: Can be integrated with these technologies for advanced perception and manufacturing.
Cultural Heritage Preservation: Significant for the replication and reconstruction of cultural relics.
Environmental Survey: The system, mounted on a remote-controlled mobile platform, can survey environments inaccessible to humans.

Key Technical Features

The ZYNQ-based LiDAR 3D modeling system incorporates several distinguishing technical features:

PCL ICP Algorithm: The system utilizes the Iterative Closest Point (ICP) algorithm from the Point Cloud Library (PCL) to perform multiple iterative calculations on LiDAR data, achieving precise point cloud registration.
Integrated Mobile Platform: The LiDAR is mounted on a Mecanum wheel robot. The robot's gyroscope and motor encoders provide real-time position and velocity information. Through coordinate system transformations, this enables precise, real-time localization of the LiDAR sensor.
Robust Indoor Reconstruction: This design enables real-time 3D reconstruction of indoor objects within a 1-5m range, with minimal blind spots. A significant advantage is its low susceptibility to lighting conditions, making it reliable in varied indoor environments.
Remote Exploration: By deploying the LiDAR on a remotely controllable robot, the system can be used to survey environments that are hazardous or inaccessible to human operators.

Key Innovations

The system introduces several innovative aspects:

Mobile App Remote Control: The Mecanum wheel robot can be controlled remotely via a mobile application, enhancing operational flexibility.
PCL-based Algorithm Foundation: The core algorithms are built upon the robust and widely-used PCL point cloud library.
Real-time Data Registration: Achieves real-time data registration through the efficient implementation of the ICP algorithm.
HLS Acceleration on PYNQ: The ICP algorithm benefits from High-Level Synthesis (HLS) acceleration on the Programmable Logic (PL) module of the PYNQ-Z2 development board, significantly boosting computational speed.

System Composition and Functionality

Overall Introduction

The system comprises three main components: the LiDAR sensor, a Mecanum wheel robot based on an STM32 microcontroller, and a Xilinx PYNQ-Z2 development board.

LiDAR Data Transmission: The LiDAR transmits collected point cloud data to the PYNQ-Z2 via an Ethernet interface.
Robot Data Acquisition: The Mecanum wheel robot is equipped with motor encoders, a gyroscope, and a Bluetooth module. Its movement and steering are controlled remotely via a mobile phone Bluetooth application. During motion, displacement and attitude information are sent to the STM32 microcontroller.
STM32 to PYNQ-Z2 Communication: The STM32 microcontroller then transmits this displacement and attitude information to the PYNQ-Z2 via a UART protocol at a baud rate of 115200, cycling in real-time.
ZYNQ Processing: The ZYNQ board calculates the LiDAR's displacement and attitude offset based on the received data. It then uses the ICP algorithm to stitch together successive point cloud data, compensating for the LiDAR's movement and orientation. The stitched point cloud data is then output via Ethernet.
Scanning Strategy: In this design, the LiDAR-equipped robot performs mobile scanning, collecting information from the left, right, and top surfaces to reconstruct the 3D environment.

Module Details

R-Fans-16 LiDAR

The system employs the R-Fans-16 navigation-grade LiDAR sensor for data acquisition. This sensor performs 360° scanning with 16 lines to achieve 3D detection and imaging. Based on high-precision laser echo signal measurement technology, the R-Fans-16 offers:

Extended Range: Detection capability up to 200m.
High Measurement Accuracy: Ranging accuracy better than 2cm.
Accurate Echo Intensity: Target reflection echo intensity is captured with 8-bit precision.
Angular Coverage: Balances pitch direction angular coverage with angular resolution. The LiDAR transmits real-time point cloud data to the PYNQ-Z2 via an Ethernet port during operation.

STM32-based Mecanum Wheel Robot

This robot platform integrates an STM32 microcontroller, which is central to its operation. For this experiment, the robot utilizes its onboard gyroscope, motor encoders, and Bluetooth module.

Sensor Data to STM32: The gyroscope and motor encoders transmit data to the STM32 microcontroller via SPI protocol.
STM32 Processing and Transmission: The microcontroller processes this data to calculate the robot's attitude and wheel speed. It then continuously sends this information to the ZYNQ via UART protocol at a baud rate of 115200.
Remote Control: The robot's movement and steering are controlled remotely using a Bluetooth-enabled mobile device.

Coordinate System Transformation

The R-Fans-16 LiDAR collects data within its own local coordinate system. The essence of 3D reconstruction in this context is to transform this data into an absolute world coordinate system, effectively converting from spherical coordinates to Cartesian coordinates.

A spherical coordinate system uses spherical coordinates (r, θ, φ) to define a point P in 3D space. Here, 'r' is the radial distance from the origin to point P, 'θ' is the polar angle between the line from the origin to P and the positive z-axis, and 'φ' is the azimuthal angle between the projection of the line onto the xy-plane and the x-axis.

The conversion formulas between spherical (r, θ, φ) and Cartesian (x, y, z) coordinates are:
x = r * sin(θ) * cos(φ)
y = r * sin(θ) * sin(φ)
z = r * cos(θ)

In this design, the robot's starting position defines the origin of the absolute world coordinate system. Subsequently, for each cycle of LiDAR data acquisition, a sub-coordinate system is established with the LiDAR as its origin. The displacement and rotation of this LiDAR-centric sub-coordinate system relative to the initial absolute coordinate system are continuously recorded.

The x, y, and z axes of the sub-coordinate system align with the LiDAR's internal axes.

Translational Information: Motor encoders measure the LiDAR's horizontal planar movement speed and direction, providing the translation offset between the LiDAR's coordinate system and the absolute coordinate system.
Rotational Information: The gyroscope measures the LiDAR's attitude angles, providing the rotational offset between the LiDAR's coordinate system and the absolute coordinate system.

By combining these measured translational and rotational values with the spherical-to-Cartesian conversion formulas, points from the LiDAR's local coordinate system can be accurately mapped into the absolute world coordinate system.

Point Cloud Registration (ICP Algorithm)

The ICP (Iterative Closest Point) algorithm is fundamental for point cloud registration, which involves aligning two point sets originating from different coordinate systems based on their geometric properties. The objective is to determine the rigid body transformation matrix (R) and translation matrix (T) that, when applied to the target point set, cause it to best overlap with the reference point set. For a target point set P and a reference point set Q, the ideal transformation is:

P' = R * P + T

Since this ideal transformation is not always perfectly achievable, the algorithm aims to minimize an objective function, typically the sum of squared distances between corresponding points:

Minimize Σ ||(R * P_i + T) - Q_i||^2

Common methods for solving for R and T include Singular Value Decomposition (SVD) and non-linear optimization. This design utilizes the SVD method.

The ICP algorithm problem is often broken down into two parts:

Coarse Registration (Global Registration): This initial step calculates the pose between two point sets to achieve a rough overlap. It provides a suitable initial value for the subsequent precise registration.
Precise Registration (Local Registration): This step applies an iterative optimization strategy to two sufficiently close point sets to achieve the final, highly accurate registration result.

Completion Status and Performance Parameters

Overview

This system successfully implements LiDAR point cloud acquisition and the collection of attitude information from the gyroscope and encoders. The PYNQ-Z2 development board's ZYNQ control chip, with its PS-PL (Processing System - Programmable Logic) design, significantly enhances the system's design convenience, feasibility, and reduces overall design complexity. The PS-PL master-slave architecture maintains design simplicity while boosting the system's operating speed and processing capabilities. The design of IP cores on the PL side dramatically accelerates algorithm computation. Specifically, the point cloud stitching component of this design benefits from PL-side IP core acceleration, improving the stitching effect and successfully achieving real-time 3D reconstruction functionality.

Completion Status

An indoor corridor environment was used for testing, featuring a row of tables, a fire extinguisher, and walls and windows on either side.

Data was collected in groups of 200 frames, stored as PCD (Point Cloud Data) files. The raw images generated from each data group are illustrated in Figure 3.2.2 (referring to the original Chinese source's image). The right side of the image clearly shows details of the table surface and legs, while the lower left side depicts the fire extinguisher. Discrete small point clouds on the right side of the image represent laser reflections from the windows projecting outdoors. If both sides of the corridor were solid walls, a more complete 3D model would be obtained.

Adjacent point cloud groups were registered using the ICP algorithm to construct a complete 3D model of the corridor as the robot traversed it.

Performance Parameters

The resolution of LiDAR point cloud acquisition decreases with increasing distance. Within the effective detection range, the measurement error for object width and depth is influenced by the stability of the equipment (i.e., the degree of LiDAR jitter).

Width Measurement Error: Within 2cm.
Inclined Object Measurement Error: 6cm.
Inclined Angle Error: 4°. It is noted that slight vibrations from the robot itself during measurement contribute to these observed errors.

Summary and Future Extensions

This ZYNQ-based LiDAR 3D modeling system demonstrates robust real-time 3D reconstruction capabilities. To further enhance its utility and user experience, several areas for expansion have been identified:

Expandability

Python Programming Integration: The current system operates on Ubuntu 18.04, with programming primarily in C++. While effective, it primarily relies on basic C language libraries and the PCL library, which can make display and interaction less convenient for users. Porting PCL to a Jupyter platform and leveraging Python would offer a more user-friendly and faster development environment.

2. Omnidirectional Stereo Scanning: The current car-mounted LiDAR setup primarily scans the left, right, and top surfaces of the environment. To obtain comprehensive volumetric information of specific objects, especially for detailed modeling, a more flexible mounting platform, such as a drone, could be employed to enable omnidirectional scanning.

【Domestic Virtual Instrument】DSP+FPGA+ADS1282-based 32-Bit High-Precision Data Acquisition Solution (II) Analog Circuit Design

Leo Liu — Mon, 01 Jun 2026 07:41:36 +0000

Originally published on the Sienovo Engineering Blog. Sienovo is the overseas brand of 深圳信迈 (Shenzhen Xinmai), building edge AI computing solutions for industrial video analytics.

This article walks through the analog front-end and ADC circuit design for a 32-bit high-precision data acquisition system built around the TI ADS1282 delta-sigma ADC, an FPGA, and a DSP. Part II focuses on the signal-conditioning chain from sensor current output all the way to the FPGA's SPI interface, covering component selection rationale, filter design, ADC operating-mode selection, pin-level interfacing, SPI timing constraints, multi-channel synchronization, and register-level control.

System Architecture Overview

Figure 4.1 shows the hardware signal block diagram. The digital signal processing board centers on two processing chips: an FPGA and a DSP. The FPGA acts as the bridge between the DSP and all peripheral interfaces and handles data pre-processing, while the DSP is the computation core.

The FPGA receives data from external devices over various data buses, performs packetization and other pre-processing, and stores results in designated on-chip memory. The DSP reads this data via its EMIF bus interface and executes the data-solving algorithms. Intermediate parameters and final results are written by the DSP back into FPGA-internal memory via EMIF, after which the FPGA transmits them externally over the CAN bus interface.

4.2 High-Precision Data Acquisition Circuit Design

Figure 4.2 shows the structural block diagram of the data acquisition module. Because the sensors output current signals while the ADS1282 requires a differential voltage input, the first task is current-to-voltage conversion. A high-precision, low-temperature-coefficient sampling resistor performs this conversion. The resulting voltage then passes through a second-order low-pass filter built from a fully-differential op-amp, which removes high-frequency noise. The filtered signal goes through clamp protection diodes before entering the ADS1282's differential inputs. The clamp diodes protect the ADC against instantaneous input overload: when the input signal level exceeds the rating of the internal ESD diodes, the external clamp diodes engage and limit the voltage, preventing damage. The ADC's digital communication interface connects directly to the FPGA, which implements an SPI bus in fabric to read the converted data.

4.2.1 Analog Signal Conditioning Circuit Design

1. Current Sampling

Accelerometer signals demand very high precision, so front-end circuit topology and component selection are critical. The op-amp must exhibit high common-mode rejection ratio (CMRR), high precision, low drift, and low noise. The low-pass filter must have excellent linearity and a well-defined cutoff frequency, and the analog front end must accept differential input.

As shown in Figure 4.3, the input current signal passes through a sampling resistor to produce a proportional voltage. The stability and accuracy of that voltage depend directly on the resistor's performance. The design uses a VISHAY VSMP-series ultra-precision metal-foil resistor, chosen for its excellent load-life stability, near-zero temperature coefficient of resistance, extremely low noise, and tight resistance tolerance.

A filter capacitor C provides additional high-frequency attenuation at the resistor output. The resulting voltage is buffered by a voltage-follower op-amp before entering the conditioning stage. The buffer is an OPA227, selected for its combination of very low noise (3 nV/√Hz), wide bandwidth (8 MHz), very high CMRR (138 dB), high open-loop gain (160 dB), low output offset voltage (75 µV max), extremely low temperature drift (0.1 µV/°C), and wide supply voltage range (±2.5 V to ±18 V). It operates across −55 °C to +125 °C.

2. Conditioning Circuit

The ADS1282 uses a differential voltage input. Differential signaling's primary advantage is strong noise immunity: external interference couples onto both signal lines nearly simultaneously and is cancelled when the difference is taken. The two signal lines also cancel each other's radiated electromagnetic fields. Differential switching transitions occur at the signal crossing point, making timing less sensitive to process and temperature variation.

Because the ADC input is differential while the signal source is single-ended, a low-noise analog conditioning circuit is needed to interface them. There are two common single-ended-to-differential conversion approaches: (1) a multi-op-amp discrete circuit, and (2) a single fully-differential op-amp IC. The discrete approach inevitably introduces more noise sources. The single-chip approach is simpler and uses fewer components, so this design uses a single fully-differential op-amp — the OPA1632 — for the front-end analog signal conditioning and filtering function.

The OPA1632 is a high-performance, low-noise fully-differential op-amp with THD as low as 0.000022% and noise density as low as 1.3 nV/√Hz. The voltage from the sampling resistor, after capacitor filtering, feeds into the OPA1632.

As shown in Figure 4.4, the OPA1632 is configured as a second-order Butterworth low-pass filter. The passband width is set by selecting appropriate resistor and capacitor values to meet design requirements. The Butterworth topology is chosen because it provides maximally flat frequency response within the passband — no ripple — while attenuating monotonically outside the passband.

Figure 4.5 shows the Multisim-simulated Bode plot for this low-pass filter. The gain curve is very flat within the passband with no ripple, and attenuation outside the passband is rapid, confirming the filter performs as designed.

4.2.2 ADC Circuit Design

Operating Mode and SNR Selection

The ADS1282 supports two primary operating modes selectable via register: low-power mode and high-performance mode. SNR varies with mode, output data rate, and on-chip PGA gain setting. Under equal conditions, high-performance mode always yields better SNR than low-power mode. Lower data rates and lower PGA gain also improve SNR.

This design selects high-performance mode at 500 SPS. The SNR at this operating point is 127 dB. Using the standard ENOB formula, this corresponds to an effective number of bits of 20.8, which meets and exceeds the system's precision requirement.

Pin Description

The ADS1282 is housed in a 28-pin TSOP SMD package. Key pin functions are as follows:

CLK: Master clock input
SCLK: Serial clock input
*DRDY*: Data-ready output; data is read on its falling edge
DOUT / DIN: Serial data output / input
MCLK, M1, M0: In modulator-output mode, MCLK is the modulator clock output and M1/M0 are modulator data outputs; otherwise these pins must be tied to fixed levels
SYNC: Synchronization input for multi-chip synchronized operation
MFLAG: Over-range flag; logic 0 = normal, logic 1 = over-range
DGND: Digital ground reference
CAPP / CAPN: Internal PGA output pins; a 10 nF capacitor must be connected between them
AINP2 / AINN2, AINP1 / AINN1: Positive/negative analog inputs for channels 2 and 1
AVDD / AVSS: Analog supply; single-ended +5 V or bipolar ±2.5 V
VREFP / VREFN: Positive/negative reference voltage inputs
*PWDN*: Power-down input, active low
*RESET*: Reset input, active low
DVDD: Digital supply, +1.8 V to +3.3 V
BYPAS: Calibration output; requires a 1 µF capacitor to DGND

1. ADC Internal Architecture and Operating Principle

The ADS1282's internal structure (Figure 4.7) consists of a multiplexer (MUX), a programmable gain amplifier (PGA), a fourth-order delta-sigma (Δ-Σ) modulator, a programmable digital filter, an over-range detection circuit, gain and offset calibration modules, a controller, and an SPI serial bus interface.

The MUX supports five configurations: channel 1 input, channel 2 input, short-circuit between the two channel inputs of the same polarity, an internal 400 Ω short-circuit test, and a common-mode test. Following the MUX is the differential-input, differential-output PGA with a gain range of 1 to 64, configured via the PGA[2:0] bits of the CONFIG1 register. The PGA drives the modulator differentially through internal 300 Ω resistors. A COG-type ceramic capacitor connected between CAPP and CAPN serves as an anti-aliasing filter, eliminating instantaneous interference generated during modulator sampling. Its corner frequency is determined by this capacitor value and the internal resistance.

The ADS1282 modulator is a high-performance, inherently stable fourth-order Σ-Δ modulator. It pushes quantization noise to frequencies well above the signal band, where the digital filter can remove it effectively. The modulator differential input signal VIN = (AINP − AINN) × PGA has a range equal to VREF = VREFP − VREFN, i.e., ±2.5 V. When the modulator input exceeds full-scale, the modulator enters a stable saturation state, the digital output code locks to the positive or negative full-scale value, and the MFLAG pin transitions from low to high, providing a hardware over-range indicator. The modulator sampling frequency is CLK/4 = 1.024 MHz in high-performance mode and CLK/8 = 512 kHz in low-power mode, where the master clock CLK is 4.096 MHz.

The modulator's bitstream output can be accessed directly or routed through the on-chip digital filter. Setting FILTR[1:0] = 00 in the CONFIG0 register puts the modulator into output mode, where M0 and M1 become modulator data outputs and MCLK becomes the modulator clock output. When not in output mode, these pins are inputs and must be tied to fixed levels.

As shown in Figure 4.8, the digital filter receives and decimates the modulator bitstream. It consists of three cascaded stages: a five-stage sinc filter with adjustable decimation ratio, a 32× decimation finite impulse response (FIR) low-pass filter with programmable phase, and a programmable first-order high-pass filter (HPF). Data can be tapped after any stage. If no on-chip filtering is desired, the raw modulator output is available. This design uses the sinc filter plus the FIR low-pass filter path with the 500 SPS filter configuration.

The digital filter output path is selected via the FILTR[1:0] bits as summarized in Table 4.2.

After the on-chip filter stages, a digital correction module consisting of an adder and a multiplier (Figure 4.9) applies offset and gain correction to the raw data according to the values stored in the OFC (offset calibration) and FSC (full-scale calibration) registers.

2. ADC Data Output Interface Circuit

The ADS1282 communicates via SPI. Conversion data is output as 32-bit two's-complement values, MSB first, with the MSB serving as the sign bit (0 = positive, 1 = negative). The SPI interface uses three core signals: SCLK, DIN, and DOUT. Data is clocked into DIN on the rising edge of SCLK and shifted out of DOUT on the falling edge. When DRDY\ goes low, a new conversion result is ready and the FPGA can initiate a read. If SCLK remains low for 64 consecutive DRDY\ periods, any ongoing data transfer or command is aborted and the SPI interface resets; the next SCLK pulse starts a new communication cycle. SCLK must be held low when idle.

Figure 4.10 shows the SPI timing requirements:

t_SCLK: SCLK period — minimum 2/f_sclk, maximum 16/f_sclk
t_SPWHL: SCLK pulse width — minimum 0.8/f_sclk, maximum 10/f_sclk
t_DIST: DIN setup time — minimum 50 ns
t_DIHD: DIN hold time — minimum 50 ns
t_DOPD: Delay from SCLK falling edge to new DOUT valid — maximum 100 ns
t_DOHD: Hold time from SCLK falling edge to DOUT change — minimum 0 ns
t_SCD_L: Interval from last rising edge of a command to first rising edge of a register read/write — minimum 24/f_sclk

Figure 4.11 shows the SPI read-data timing. Each data packet is four 8-bit bytes (32 bits total), transmitted MSB first out of the DOUT pin.

Multi-Channel Synchronization

This system uses three ADS1282 channels that must operate synchronously to avoid inter-channel timing errors. Synchronization can be achieved either through the SYNC pin or by issuing a sync command over SPI. When a sync event occurs, the ADC resets its internal memory and begins a new conversion. The ADS1282 supports two sync modes:

(1) Single-pulse sync mode. The ADC synchronizes only once during operation. Synchronization occurs at the first CLK rising edge after the SYNC rising edge, or at the eighth SCLK rising edge after a sync command. After this single sync event, the ADC continues converting at the selected rate without further synchronization.

(2) Continuous sync mode. The ADC re-synchronizes on every SYNC rising edge, performing one conversion per sync pulse.

This design uses SYNC-pin single-pulse synchronization. After all three ADC channels synchronize via the shared SYNC pin, they continue converting at the same rate continuously. Figure 4.12 shows the wiring of the three-channel ADC array. Each ADC's interface carries CLK, SYNC, SCLK, DIN, DOUT, and DRDY. Importantly, CLK and SYNC originate from the same internal FPGA signal, ensuring tight, deterministic synchronization across all three channels.

Register Access Protocol

All ADS1282 control is exercised by sending commands over SPI, including register read and write operations. Figures 4.13 and 4.14 show the read and write register timing diagrams. A command word must be sent first; it encodes the starting register address and the number of registers to access (Table 4.3). The register data transfer follows immediately after the command word is clocked in.

4.2.3 Voltage Reference Design

A stable, low-noise voltage reference is essential for achieving the full dynamic range of a 32-bit ADC. The reference voltage VREF = VREFP − VREFN defines the modulator's input span of ±2.5 V, so any noise or drift on the reference directly adds to the ADC's output noise floor and degrades accuracy. The reference circuit design — covered in the next section — must match the precision of the sampling resistor, the OPA1632 conditioning stage, and the ADS1282 itself to avoid becoming the system bottleneck.

Design and Implementation of an MPSoC-Based Hardware Platform for Autonomous Driving Multi-Sensor Fusion Localization

Leo Liu — Sun, 31 May 2026 07:16:19 +0000

Originally published on the Sienovo Engineering Blog. Sienovo is the overseas brand of 深圳信迈 (Shenzhen Xinmai), building edge AI computing solutions for industrial video analytics.

1. Introduction

High-precision localization is a core prerequisite for environmental perception, path planning, and vehicle motion control in autonomous driving. The accuracy and timing synchronization performance of localization directly determine the reliability and safety of autonomous driving systems. Currently, mainstream localization solutions mostly adopt a discrete multi-module architecture: multi-sensor data acquisition relies on independent hardware boards, while positioning computation and algorithm acceleration depend on industrial PCs, GPUs, or high-end CPUs.

This separated architecture has several engineering drawbacks: hardware interconnection among multiple modules results in low system integration and bulky structure; inter-board communication introduces transmission latency, making it difficult to achieve microsecond-level time synchronization across sensors; general-purpose computing platforms tend to have high power consumption and heat dissipation, failing to meet automotive-grade requirements for wide temperature range, low power, and reliability.

MPSoC integrates ARM processors and programmable FPGA logic on a single chip, enabling hardware-software cooperative parallel computing. The FPGA excels at high-precision timing control, synchronized multi-sensor data acquisition, and hardware acceleration of algorithms, while the ARM is adept at system scheduling, data fusion, and task management. Building an integrated localization hardware platform based on MPSoC enables tight integration of synchronized sensor acquisition, time reference establishment, and positioning algorithm acceleration—offering an effective technical solution to current hardware bottlenecks in autonomous driving localization.

Based on multi-sensor localization theory and the characteristics of MPSoC heterogeneous architecture, this article presents the complete design of a high-precision localization hardware platform, including system architecture, modular circuit development, embedded software porting, and functional performance testing. Combined with industrial engineering practices, it also discusses the customization design methodology and key implementation considerations for MPSoC-based autonomous driving localization motherboards, providing valuable reference for similar navigation and positioning hardware development.

2. Overall Scheme of Multi-Sensor Localization for Autonomous Driving

2.1 System Sensor Composition

The localization system adopts a GNSS + IMU + 3D LiDAR + wheel encoder multi-source fusion architecture, with complementary functions among sensors:

GNSS: Provides global absolute position information and outputs PPS (pulse-per-second) and NMEA timing data, serving as the primary time reference for the system;
IMU (Inertial Measurement Unit): Outputs high-frequency tri-axis acceleration and angular velocity data, compensating for positioning outages during satellite signal blockage;
3D LiDAR: Captures real-time environmental point cloud data, enabling high-precision relative localization through map matching;
Wheel Encoder: Outputs wheel rotation pulse signals for dead reckoning and vehicle motion state verification.

Time synchronization and spatial registration across sensors are prerequisites for fusion localization. Only with temporally aligned sensor data can the accuracy and robustness of integrated positioning be effectively improved.

2.2 MPSoC Heterogeneous Task Allocation

The system adopts a collaborative architecture between PS (Processing System) and PL (Programmable Logic), following the design principle of "real-time tasks in hardware, complex tasks in software":

PL (FPGA): Establishes a high-precision system time reference, parses GNSS PPS pulses, and calibrates clock signals using a high-stability oscillator; performs hardware-synchronized acquisition of IMU and encoder data; handles serial timing control, data buffering, and AXI bus data exchange; implements pipeline-based hardware acceleration of positioning algorithms.
PS (ARM): Runs an embedded Linux system to manage task scheduling and peripherals; receives and parses LiDAR point cloud data; performs multi-sensor data fusion and positioning computation; handles host communication, parameter configuration, and data logging.

2.3 Overall Platform Architecture Diagram

The system adopts a layered architecture: Sensor Layer → Hardware Interface Layer → MPSoC Heterogeneous Processing Layer → Application Computation Layer. All sensor signals are conditioned and fed into the MPSoC, where the PL handles synchronized timing and raw data acquisition, and the PS performs algorithm computation and outputs positioning results. This compact, streamlined architecture avoids timing delays and electromagnetic interference issues inherent in discrete designs.

3. Overall Hardware Platform Design

3.1 Main Control Chip Selection

The platform uses the Xilinx Zynq UltraScale+ series MPSoC as the main controller, while remaining compatible with industrial-grade heterogeneous processors such as Zynq7000 and TI AM5728/AM62x. The chip integrates multi-core ARM A53 and R5 real-time processing units, along with abundant programmable logic resources. It natively supports DDR4, Gigabit Ethernet, USB 3.0, mSATA, and high-speed serial interfaces, operating across a temperature range of -40°C to +85°C, meeting long-term stability requirements for automotive and industrial unmanned equipment.

3.2 Modular Hardware Design

The hardware platform employs a modular, layered design, divided into five functional units:

Minimal System Module: Includes the MPSoC controller, DDR4 high-speed memory, eMMC solid-state storage, configuration circuitry, high-stability oscillator, and real-time clock unit, providing a stable operating foundation and timing reference.
Sensor Interface Module: Features dedicated signal conditioning circuits for IMU, GNSS, LiDAR, and encoder, performing differential level conversion, signal shaping, and isolation protection to comply with each sensor's electrical interface standards.
Power Management Module: Supports 20–36V wide-range automotive input, integrates overvoltage, undervoltage, and reverse-polarity protection, and implements multi-channel DC/DC voltage conversion following the MPSoC power-up sequence to deliver stable 1.2V, 1.8V, 3.3V, and 5V outputs for reliable module startup.
High-Speed Communication and Storage Module: Equipped with dual Gigabit Ethernet ports, USB 3.0, mSATA, and SD card interfaces to meet demands for high-bandwidth LiDAR data transmission, system image storage, and positioning log recording.
Debug and Expansion Module: Exposes JTAG and serial debug interfaces, status indicator LEDs, and reserved GPIO expansion pins for convenient hardware debugging, firmware updates, and functional extensions.

3.3 High-Precision Time Reference Design

The system employs a combined GNSS PPS pulse and high-stability oscillator timing scheme: leveraging the long-term stability of GNSS timing and the short-term low-drift characteristics of the oscillator, a microsecond-precision timer is implemented in the PL. Under normal conditions, the local clock is continuously calibrated by the PPS pulse; during GNSS signal outages, the high-stability oscillator maintains the system time reference.

For different sensor types, three synchronization modes are applied: active synchronization, passive synchronization, and timing synchronization. The IMU uses trigger-based active synchronization, the wheel encoder uses pulse-based passive synchronization, and the LiDAR uses PPS-based timing synchronization—achieving unified timing across all sensors with microsecond-level synchronization accuracy.

4. Core Circuit Module Design

4.1 Power Circuit Design

The power circuit employs staged protection and sequenced power delivery. EMI filtering and reverse-polarity protection are implemented at the front end, while multi-channel power regulators generate core and peripheral voltages at the back end. The MPSoC power-up sequence is strictly followed to prevent chip lockup due to improper sequencing. Load current limiting and thermal protection are also added to adapt to complex automotive electrical environments.

4.2 Sensor Interface Circuit

The IMU uses an RS422 differential interface, adapted to FPGA I/O levels via level-shifting chips to ensure noise immunity over long-distance transmission. The GNSS interface reserves independent serial and PPS lines with electrical isolation to prevent timing signal distortion. Differential encoder pulses are shaped and filtered using Schmitt triggers to suppress high-frequency noise. The LiDAR connects directly to a Gigabit PHY chip to meet real-time, high-bandwidth point cloud transmission requirements.

4.3 High-Speed Interface Circuit

Industrial-grade PHY devices are used for Gigabit Ethernet, with RGMII routing rules followed, including impedance matching and length-matched traces to ensure high-speed communication stability. DDR4 memory is designed strictly according to high-speed PCB guidelines, controlling trace impedance, length, and spacing for reliable large-data read/write operations. ESD protection components are added to mSATA and USB 3.0 interfaces to enhance robustness in industrial environments.

5. Embedded Software Design and Porting

5.1 Linux System Customization and Porting

Using the Petalinux development environment, the embedded Linux system is customized and trimmed. Device trees are adapted to hardware peripherals, kernel drivers, file systems, and boot methods are configured, supporting dual boot modes from SD card and QSPI flash. PS-PL communication protocols are set up, and a multi-core task scheduling framework is established to meet real-time algorithm execution requirements.

5.2 FPGA PL Logic Design

Verilog HDL is used for logic development, primarily implementing: high-precision time counter and PPS pulse calibration logic; state-machine-based timing control for multi-sensor data acquisition; serial command parsing and FIFO data buffering; AXI bus protocol adaptation for high-speed data transfer from PL to PS. The design follows a modular state-machine approach with efficient resource utilization and good timing convergence.

5.3 ARM PS Application Software Design

The PS software adopts a layered architecture: the lower layer includes drivers for serial, Ethernet, and sensors; the middle layer handles data parsing and temporal alignment; the upper layer implements multi-sensor fusion algorithms, positioning computation, host interaction, and data storage. The software uses time-sliced multitasking to ensure real-time performance of positioning tasks, while supporting online parameter configuration and offline log storage.

6. System Function and Performance Testing

6.1 Basic Function Testing

A static lab test platform was built, integrating LiDAR, IMU, GNSS, and wheel encoder for full-system integration testing. Results show all hardware interfaces function correctly, PS-PL command interaction is reliable, and all sensors can power up synchronously and stably acquire raw data without packet loss or timing anomalies. The system demonstrates stable hardware and software operation.

6.2 Time Synchronization Accuracy Testing

Using an oscilloscope to compare the edge deviation between the GNSS standard PPS and the system's local clock signal, the time synchronization error remains within microsecond level under normal satellite lock conditions. During continuous 4-hour operation without GNSS signals, the cumulative time error is less than 140μs, maintaining high relative timing precision—meeting the requirements for multi-sensor fusion localization.

6.3 Algorithm Acceleration Performance Testing

Using LiDAR point cloud feature extraction as the test algorithm, performance was compared across the MPSoC platform, an embedded GPU platform, and a desktop CPU platform. Results show that the MPSoC, leveraging FPGA pipeline acceleration, achieves computational efficiency close to that of a desktop CPU, while consuming significantly less power than general-purpose computing platforms. This demonstrates dual advantages of low power and high computing performance, making it ideal for embedded deployment in unmanned systems.

6.4 Platform Advantages

High Integration: A single MPSoC chip integrates data acquisition, timing, computation, and acceleration, eliminating the need for multiple discrete boards and reducing device size;
High Timing Precision: Microsecond-level multi-sensor time synchronization significantly improves fusion localization accuracy;
Low Power and Automotive-Grade Compatibility: Heterogeneous architecture enables on-demand compute allocation, with wide-temperature and noise-resistant design suitable for automotive and outdoor unmanned applications;
Customizable Hardware and Software: PL logic, PS software, and peripheral interfaces can be tailored to project needs;
Easy Algorithm Iteration: Positioning algorithms run on the ARM side, allowing upgrades and optimization without modifying hardware logic.

7. Engineering Practice of Custom MPSoC Autonomous Driving Localization Motherboard

From an industrialization perspective, devices such as autonomous vehicles, AGV inspection robots, agricultural machinery, and power line inspection robots all require customized MPSoC hardware platforms. Drawing on years of industrial motherboard development experience, Sienovo offers end-to-end customization services—from solution evaluation and schematic/PCB design to software/hardware development and mass production—based on full-series MPSoC chips from Xilinx, TI, and Intel.

Custom Chip Solutions: Core and full-board designs can be tailored to Zynq 7000/UltraScale+, TI AM5728/AM62x, and other heterogeneous platforms based on positioning accuracy, compute needs, and cost constraints.
Customizable Interfaces: Dedicated peripherals such as Gigabit LiDAR ports, differential serial interfaces for IMU/GNSS, encoder interfaces, CAN/RS485, 5G expansion, and multiple Ethernet ports can be customized to match diverse sensing and networking requirements.
Industrial-Grade Reliability: Designs support -40°C to +85°C wide temperature range, shock resistance, triple protection (moisture, dust, corrosion), dual power redundancy, and EMC compliance, meeting automotive and industrial safety standards.
Turnkey Hardware and Software Delivery: Services include Linux/Petalinux system porting, low-level driver development, PL logic design, and multi-sensor synchronization framework integration—enabling customers to focus on high-level positioning algorithms and application development, significantly shortening R&D cycles.

8. Conclusion

This article presents the design, circuit development, software porting, and performance testing of an MPSoC-based multi-sensor fusion localization hardware platform for autonomous driving. It addresses engineering challenges of traditional discrete platforms—low integration, poor timing synchronization, and high power consumption. By leveraging FPGA's real-time timing control and ARM's flexible algorithm scheduling, the platform achieves microsecond-level synchronized sensor acquisition and hardware-accelerated positioning algorithms, meeting stability and accuracy requirements for autonomous driving and intelligent navigation systems.

As autonomous driving and outdoor unmanned equipment move toward industrialization, customized MPSoC industrial motherboards have become the mainstream choice in hardware development. The standardized core board + custom baseboard development model enables rapid adaptation to multi-sensor localization needs across diverse scenarios, providing reliable support for cost reduction, efficiency improvement, and shortened time-to-market for unmanned systems.

Sienovo provides MPSoC customization and manufacturing services.

RK3399+FPGA-based Ground Test Bench Multi-parameter Data Logger Solution (Part 1) Software Design and Testing

Leo Liu — Sat, 30 May 2026 06:58:19 +0000

Originally published on the Sienovo Engineering Blog. Sienovo is the overseas brand of 深圳信迈 (Shenzhen Xinmai), building edge AI computing solutions for industrial video analytics.

This blog post delves into the software design and testing aspects of an RK3399+FPGA-based ground test bench, specifically focusing on a multi-parameter data logger solution. We will explore the software's architecture, its core functionalities, the underlying Qt communication mechanisms, and the practical implementation and testing of its user interface, providing a comprehensive overview for engineers working with similar embedded systems and data acquisition challenges.

Software Design Principles for the Ground Test Bench

The development of the test bench software began with a thorough analysis of the operational requirements for a specific type of data logger. The primary goal was to create a fully functional, user-friendly, and intuitive test bench application using Linux-Qt as the main development framework.

The software's architecture is structured around three distinct operating modes, each managed by its own functional window:

Analog Signal Source: Responsible for generating simulated data.
Memory Detection and Readback: Manages the retrieval of stored data.
Data Processing: Handles the analysis and separation of raw data.

These windows are designed to function independently, allowing for focused testing of specific modules, yet they are also closely integrated to support comprehensive, end-to-end testing workflows.

Analog Signal Source Window

The Analog Signal Source window is crucial for pre-flight testing, simulating various ground data sources. The host computer sends commands to the main CPU within the test bench, which then generates a diverse set of signals:

5 channels of switching quantities (digital on/off signals)
2 channels of analog quantities
1 channel of 422 data (RS-422 serial communication)
5 channels of network data

This simulation allows for a complete system test before actual flight experiments. Pre-configured analog data is sent and stored in Flash memory, then read back. This data undergoes initial comparison and further processing to verify the entire system's operational status.

Memory Detection and Readback Window

This window is dedicated to managing and retrieving data stored in the data logger's Flash memory. Its functionalities include:

Network configuration
Memory selection (e.g., primary or backup storage)
Querying current storage status
Setting read counts and read volumes
Stopping read operations
Erasing the data logger's memory

The core task here is to efficiently read back existing data from the Flash memory.

Data Processing Window

The Data Processing window's main function is to separate raw data files that have been read back and saved to the desktop. It allows for the segregation of data according to different channels, facilitating independent analysis and research.

Communication Modes

The system employs two primary communication modes:

Serial Command Transmission and Reception: This mode uses a send-and-feedback mechanism to ensure the accuracy of each command. Real-time feedback is displayed on the software interface, and a .log file is generated upon exit for easy review. The lower-level device provides immediate feedback upon receiving a command.
UDP Data Readback: When the host computer sends a data readback command, the data logger transmits the data stored in Flash memory back to the host via UDP over the network. This data is then saved as a local file for subsequent processing. The data packet format is structured as: Frame Header + Frame Count + Data + Checksum.

Detailed control command tables are used to define the specific instructions for interaction between the host and the data logger.

Software Function Overview

The software incorporates the following key functionalities:

Work Mode Selection: Users can choose between "Analog Source Mode," "Data Readback Mode," and "Data Separation Mode" directly from the software's startup interface.
Analog Source Control: In Analog Source Mode, the software sends serial commands to enable relays on the test bench and control the CPU to generate simulated data. Operations can be performed in either automatic or manual modes. During experiments, any switching quantity can be enabled or disabled at any time, which is useful for verifying the proper functioning of specific channels within the system.
Data Readback Management: In Data Readback Mode, the host computer queries storage information from a specified memory unit to select data for reading. No other operations are permitted during the read process. Upon completion, users can choose to erase the data logging device (a full erase), which requires a re-confirmation step.
Data Separation: The Data Separation Mode allows users to further process read-back and stored data files. Users can separate one or multiple data channels as needed. Each data separation operation functions as an independent module, ensuring no interference between different channel separations.

Qt Signal Communication Mechanism

As a host computer control software, its primary function is to communicate with lower-level devices through command transmission and reception. When developing with Qt, communication is a fundamental consideration. Qt's unique "Signal & Slot" mechanism is a powerful communication paradigm. Conceptually, a signal is like a switch, and a slot function is like a light bulb. When the switch is triggered, the light bulb activates. Similarly, when an event associated with a signal function occurs (e.g., clicking a button), a signal is emitted. The purpose is for a corresponding slot function to execute. This mechanism abstracts away complex low-level implementations; once a signal and slot are connected, the signal emitter doesn't need to know how Qt finds and executes the slot function. Compared to callback functions in some other development frameworks, Qt's Signal & Slot mechanism is more flexible and makes programming goals for various UI components clearer.

Signals and slot functions are connected using the QObject::connect() function, typically written as:
QObject::connect(sender, SIGNAL(signal()), receiver, SLOT(slot()));
or
connect(sender, SIGNAL(signal()), receiver, SLOT(slot())); (where QObject can be omitted).

In this function:

sender refers to the name of the control that emits the signal.
signal() is the name of the signal itself (e.g., a button click, text input).
receiver refers to the name of the class that receives the signal (e.g., a window, a dialog).
slot() is the corresponding slot function, defining the operation the receiving class should perform (e.g., popping up a window, displaying text).

When programming with the Signal & Slot mechanism, several key points should be understood:

One-to-Many and Many-to-One Connections: A single signal can be connected to multiple slot functions, and multiple signals can also be associated with the same slot function. There isn't a fixed numerical match between signals and slots. When a signal is connected to several slot functions, their execution order depends on the specific program logic and their connection sequence. This one-to-many and many-to-one relationship can be visualized as shown in Figure 3.2.
Signal-to-Signal Connections: Signals can also be connected to other signals. A signal doesn't strictly have to correspond to a slot function; it can trigger another signal, which then responds to one or more slot functions. This offers great flexibility depending on the specific application requirements.
Immediate and Synchronous Execution: The association between signals and slot functions is immediate and synchronous. Similar to an interrupt, when a signal triggers a slot function, it executes immediately. The main program flow will only resume after the slot function has completed its execution.

Beyond the fundamental Signal & Slot mechanism, Qt also provides functions to handle other event triggers, such as mouse operations and keyboard input. These operations have corresponding functions in Qt to respond to them. Figure 3.3 illustrates some common event handling functions.

Software Program Interface Design and Implementation

Based on Qt's communication mechanisms and project requirements, the software's user interface is designed with a startup screen and three main functional windows: the Analog Source window, the Data Readback window, and the Data Separation window.

Boot Interface

The boot interface displays the software name, version number, and provides entry points to the three main functional windows. Each window is functionally independent and does not interfere with the others. Users can select the appropriate function button based on their specific needs.

Ground Data Simulation Window

Ground data simulation is an essential part of the overall system, necessary for verifying the data logger's operation before flight experiments. The simulation source offers both manual and automatic modes.

Upon clicking "System Power On," the software sends a command to the CPU, activating the relays and powering up the system. In manual mode, users can freely select any switching or analog quantity for testing, which is convenient for diagnosing faults in specific system channels. In automatic mode, the software sends a single command, and the CPU automatically powers up, sequentially activating various switching and analog data channels, recording them in memory, thereby automating the operation. The test bench features corresponding indicator lights to show whether each switch is open. The right-hand panel of the window displays real-time operation and feedback information.

The working duration for the analog source mode can be controlled by the user, typically set for 3-5 minutes. The on/off times for each switching quantity are stored as data in memory, allowing for a direct visual representation of each switch's activation time and sequence upon readback. Buttons like "System Power On" and "Automatic Mode" are designed to be toggle buttons; a single click changes their label to "System Power Off" and "Stop Test," respectively. The main CPU on the test bench is responsible for overall data framing and storage of received data.

Data Readback Window

The Data Readback window is a core component of the software's functionality, responsible for retrieving framed data stored in Flash memory back to the host computer. The hardware interface utilizes a 100-megabit Ethernet chip (W5300) and employs the UDP protocol for data readback. The retrieved data is saved to the desktop in .hex file format.

The readback process involves several steps:

Click the "System Power On" button.
Configure the network IP address and port number.
Click the "Connect" button; the interface will display the current network connection status.
Select the memory unit. The system is equipped with two memory chips, both with backups, ensuring data integrity.
Click the "Read Recorder Information" button to display the current storage status on the interface, including power-on count and data size. Users can then select the number of reads and the read volume based on this information.
Click "Read Main Memory" or "Read Backup Memory" to begin reading. The data will be saved to the desktop, named according to the read time. A prompt will appear in the text box upon completion.
Finally, click the "Stop Reading" button to free up the FPGA. Users can also choose to erase the memory as needed.

The right-hand text box of the interface provides operational steps, and every operation and feedback is displayed in real-time in the lower text box. To prevent user errors, each button is grayed out and unusable until the previous operation is completed. After reading is finished, clicking "Exit" returns to the main interface for subsequent operations.

Data Separation Window

The Data Separation window's primary function is to process and separate the raw data files that have been read back.

Each button in the interface represents the separation of a single data channel. Clicking a button automatically opens a file index window, allowing the user to select any previously read raw data file for separation. Upon completion, an information prompt indicates that the separation is finished. The output sub-data files are saved to the desktop. By separating the data, users can further observe and analyze each channel's data, compare it with pre-set simulated data, and draw conclusions about the overall system's operation.

The fundamental idea behind data separation is to segregate data based on the frame headers of different channels. Since data from each channel is independent and unaffected by others, and the data format is fixed (Frame Header + Frame Count + Data + Checksum), using the frame header for judgment allows for accurate separation of the raw data. The program design flowchart for the separation process is illustrated in Figure 3.8.

Sienovo (Sienovo) provides the RK3399+FPGA hardware and software solution for this system.

Software Function Testing

To validate the software, the RK3399 development board was connected to the test bench. This included providing 12V power from the test bench to the development board, and connecting the serial and network ports to the test bench. Upon powering on the test bench, the voltage display showed a stable 27.7V, and the indicator lights for power, serial, and network connections were all green, confirming proper hardware connectivity.

Testing Analog Source Mode

After launching the software, the "Analog Source" work mode was selected. In "Manual Mode," various switching and analog quantity buttons were clicked sequentially, with the working time set to 5 minutes. The software interface displayed real-time operation information, confirming the commands were being sent and processed correctly.

Testing Data Readback Mode

Once the system's simulation work concluded, the data readback process began. In the software's "Data Readback" interface, the read configuration was performed according to the prompts. This involved setting up network parameters and initiating the read operation.

Upon successful readback, the data was automatically saved to the desktop as a source file, named according to the storage time.

Test Results

The successfully read-back data was transferred from the RK3399's USB port to a PC. Using data analysis software, including Hexedit and Matlab, the raw data was processed and separated based on the data frame headers for each channel. This resulted in the output of 7 distinct data channels. By viewing and processing these channels, corresponding data source code and waveform diagrams were obtained. It was observed that within the first 8 bytes of each data channel, 4 bytes represented the data frame header, and the subsequent 4 bytes represented the frame count, both clearly visible in the raw data.

The test results for switching and analog quantities are shown in Figure 5.4. For this specific test, the data frame header was F1F20000, the frequency was 100Hz, and the frame count interval was 2.

DEV Community: Leo Liu

Autonomous Obstacle Avoidance System for UAVs Based on OMAPL138 DSP+ARM+FPGA, Millimeter-Wave Radar, and Monocular Vision Fusion

(3) The integration of DSP and ARM on a single SOC simplifies hardware design, lowering product power consumption and hardware costs.

Adapter Pattern and Facade Pattern (Highly Recommended)

Microcontroller Communication Protocol Design

ARM+FPGA-based Industrial Medical Endoscope Solution

Introduction

1. Background and Motivation

1.1 Medical Device Context

1.2 FPGA in Medical Imaging

2. FPGA Architecture and Design Flow

2.1 How FPGAs Implement Logic

2.2 Design Flow

3. System Design

3.1 Design Task

3.2 Algorithm Selection

3.3 Key Technical Challenges

3.4 Solutions

4. Hardware Module Overview

4.1 Power Module

4.2 Acquisition Module (OV7670 Camera)

4.3 Buffer Module (SDRAM)

4.4 Display Module (VGA)

What's Next

Virtual Functions in C++

Virtual Functions in C++ (1)

VXWORKS Kernel Analysis

T1, T2, and T3 are high, medium, and low priority tasks, respectively. T3 acquires a related resource by owning a semaphore. When T1 preempts T3 and requests the same

Detailed Explanation of ANSI C Standard File I/O Functions

I. Overview

II. Function Details

Power Data Acquisition and Transmission Design Based on OMAPL138 + Xilinx Spartan-6

1 Evaluation Board Overview

ZYNQ Lidar-Visual SLAM Algorithm Porting and Design

FPGA-Accelerated Stereo Vision on ZYNQ

The Broader Context of 3D Reconstruction

Sienovo's ZYNQ-based LiDAR 3D Modeling System

Application Areas

Key Technical Features

Key Innovations

System Composition and Functionality

Overall Introduction

Module Details

R-Fans-16 LiDAR

STM32-based Mecanum Wheel Robot

Coordinate System Transformation

Point Cloud Registration (ICP Algorithm)

Completion Status and Performance Parameters

Overview

Completion Status

Performance Parameters

Summary and Future Extensions

Expandability

【Domestic Virtual Instrument】DSP+FPGA+ADS1282-based 32-Bit High-Precision Data Acquisition Solution (II) Analog Circuit Design

System Architecture Overview

4.2 High-Precision Data Acquisition Circuit Design

4.2.1 Analog Signal Conditioning Circuit Design

1. Current Sampling

2. Conditioning Circuit

4.2.2 ADC Circuit Design

Operating Mode and SNR Selection

Pin Description

1. ADC Internal Architecture and Operating Principle

2. ADC Data Output Interface Circuit

Multi-Channel Synchronization

Register Access Protocol

4.2.3 Voltage Reference Design

Design and Implementation of an MPSoC-Based Hardware Platform for Autonomous Driving Multi-Sensor Fusion Localization

1. Introduction

2. Overall Scheme of Multi-Sensor Localization for Autonomous Driving

2.1 System Sensor Composition

2.2 MPSoC Heterogeneous Task Allocation

2.3 Overall Platform Architecture Diagram

3. Overall Hardware Platform Design

3.1 Main Control Chip Selection

3.2 Modular Hardware Design

3.3 High-Precision Time Reference Design

4. Core Circuit Module Design

4.1 Power Circuit Design

4.2 Sensor Interface Circuit