DEV Community: Leo Liu

Linux Process Switching and Return Values of Kernel Threads

Leo Liu — Fri, 29 May 2026 07:16:53 +0000

Originally published on the Sienovo Engineering Blog. Sienovo is the overseas brand of 深圳信迈 (Shenzhen Xinmai), building edge AI computing solutions for industrial video analytics.

The process in Linux is a fundamental concept. A process has two starting points when transitioning from the run queue to execution: one is label 1 in the switch_to macro: "1:\t", and the other is ret_from_fork. Almost all processes that are not newly created start from label 1 mentioned above. The switch_to macro is a mandatory path for all processes except the kernel itself to go through in order to run. Thus, although the Linux process system and scheduling mechanism are highly complex, the overall structure resembles an hourglass, with the switch_to macro representing the narrowest point in the middle. To move from one end to the other, every process must pass through this point. For non-newly-created processes, execution always begins at label 1. Let's first examine how this works:

#define switch_to(prev,next,last) do {                            \
         unsigned long esi,edi;                                   \
         asm volatile("pushfl\n\t"                                 \
                     "pushl %%ebp\n\t"                            \
                     "movl %%esp,%0\n\t"        /* save ESP */    \
                     "movl %5,%%esp\n\t"        /* restore ESP */ \
                     /* Note: the kernel stack has now switched. Hence, local variables on the original stack become invalid. To preserve their values, they must be saved. For efficiency, 'prev' is stored in a register for later use */ \
                     "movl $1f,%1\n\t"          /* save EIP */    \
                     /* Here, any process previously switched out will return with label 1 as its eip */ \
                     "pushl %6\n\t"             /* restore EIP */ \
                     /* Push the new process's eip onto the stack. Since the next instruction is a jmp, and the called function ends with a return, the return semantics allow eip to be popped from the stack into the eip register. In fact, this jmp to __switch_to acts like a manual call — a clever trick */ \
                     "jmp __switch_to\n"                          \
                     "1:\t"                                       \
                     /* The instruction at label 1 is simple, yet this simplicity enables architectural elegance */ \
                     "popl %%ebp\n\t"                             \
                     "popfl"                                      \
                     :"=m" (prev->thread.esp),"=m" (prev->thread.eip), \
                      "=a" (last),"=S" (esi),"=D" (edi)            \
                     :"m" (next->thread.esp),"m" (next->thread.eip), \
                      "2" (prev), "d" (next));                    \
} while (0)

Linux implements this single-point switching mechanism to reduce complexity. In fact, many operating system kernels adopt a similar approach. The "single point" here does not refer solely to the switch_to macro, but specifically to saving and restoring the eip register, ensuring that all processes resume execution from the same location upon rescheduling. However, there is a minor imperfection: Linux does not make all processes start from label 1 when transitioning from ready to running. Examining the implementation of do_fork reveals that newly created processes do not follow this rule — their eip is set to ret_from_fork, not label 1. Why is this?

When creating a new process, a starting address must be manually specified — after all, execution must begin somewhere. But where should this starting point be? (Do not confuse this with regs.eip, which represents the normal execution eip of a process. regs.eip belongs to the process state saved during a system call, whereas the starting point here is used by the kernel for process management and is unrelated to the process or kernel thread itself.) Ideally, the new process should be made to appear as if it were resuming like any other existing process — promoting uniformity and simplifying management. In that case, why not set this starting address to label 1?

But where exactly is label 1 located? Is the embedded assembly macro making the address of label 1 difficult to obtain? If so, the label could simply be moved outside the macro into a standalone location, allowing both existing and newly created processes to begin execution from that fixed address. Surely the kernel designers are not less intelligent than I am. However, such an indirection would incur extra instruction-fetching overhead in time and space — direct use of an inline assembly label is more efficient. Moreover, using ret_from_fork achieves the same effect as starting from label 1. Let's examine the design of the process switch function.

Existing processes enter switch_to via schedule, eventually reaching label 1. After switch_to, only finish_task_switch and a check for the rescheduling flag remain. Now consider ret_from_fork:

ENTRY(ret_from_fork)
        pushl %eax      // Note: returning from __switch_to called by switch_to, which returns 'prev' in %eax. This push passes 'prev' as the argument to schedule_tail.
        call schedule_tail
        GET_THREAD_INFO(%ebp)
        popl %eax
        jmp syscall_exit

As seen above, ret_from_fork calls schedule_tail with the outgoing process (prev) as the argument. schedule_tail immediately calls finish_task_switch, aligning the logic with what follows switch_to in schedule. The parameter handling is also correct. What about the logic after finish_task_switch, such as checking the reschedule flag? That is handled in syscall_exit called by ret_from_fork, which checks whether rescheduling is needed and enters the normal schedule flow if so — perfectly correct.

In fact, the need for finish_task_switch cleanup complicates things slightly, but its design is quite clever. It checks whether the previous process still needs to exist. If it's already dead, finish_task_switch releases its task_struct. The value of prev must be preserved because prev is a local variable in schedule, stored on the previous process's kernel stack. After switching to the new kernel stack (note that schedule uses two kernel stacks), prev becomes inaccessible. Hence, it must be saved.

Even if a process being exited has no remaining references, its task_struct cannot be freed immediately in do_exit. Linux lacks a dedicated scheduler manager that could detect this and automatically switch to another process. Ultimately, the exiting process must call schedule itself. When it does, it becomes current, which then becomes prev. The entire switching process depends on the exiting process's task_struct. Only after switching to the new process can the now-unused task_struct of the exiting process be safely freed.

This design of process exit is elegant. Although the absence of a dedicated scheduling management thread may seem inelegant at first glance, Linux is not a microkernel. The monolithic kernel's advantage lies in efficiency. Letting the process that needs to switch call the switch code directly, and having other ready processes signal the currently running process to initiate scheduling, is clearly the most efficient approach. Introducing a scheduling management thread would require sending notifications for every scheduling event, making many switches inefficient — albeit more aesthetically pleasing. In this respect, Linux's scheduling is a harmonious, self-organized, preemptive collaboration, whereas kernels with a scheduler manager enforce rigid control.

asmlinkage void schedule_tail(task_t *prev)
{
        finish_task_switch(prev);
        if (current->set_child_tid)
                put_user(current->pid, current->set_child_tid);
}

At this point, Linux process switching is still within the kernel's process management code and has not yet begun user-process-related actions. That is, the registers saved in regs have not yet taken effect. Only when the kernel determines that all internal tasks are complete and nothing is left will it begin the actual work of the process — the logic of RESTORE_ALL.

Newly created processes use loosely structured code to align with the compact logic of schedule. Once such a process starts, it enters the large Linux process-switching hourglass and follows the standard single-point switching flow.

Finally, let's examine the return value of kernel threads — specifically, the return value of kernel_thread. One could argue that kernel threads should not have been designed this way at all. The implementation of kernel_thread shows that the kernel essentially reuses the mechanism for creating user processes to create kernel threads. In user space, process creation uses copy-on-write semantics, but kernel threads do not. Moreover, Unix process creation involves duplicating the parent's address space without any special strategy for the child. Special behaviors must be set later via exec or other means. How a child process runs is determined in the parent's source code by checking the return value of the fork function.

The return value of the user-space fork function is crucial because parent and child share an address space (with copy-on-write), and the return value distinguishes between them. Although kernel threads are also implemented via do_fork, the child's execution function — its behavioral strategy — is specified from the start. Thus, the return value of do_fork becomes less important. In fact, when creating a kernel thread in the kernel, it never returns 0, nor is there a concept of "returning 0 means child process." Even in user space, the 0 returned by fork is not from do_fork — do_fork only returns the new process's PID. The 0 is instead popped into eax during RESTORE_ALL, just before returning to user space from ret_from_fork. The library implementation of fork then uses eax as the return value. In reality, the child process never passes through do_fork when entering user space. Its thread.eip is set to ret_from_fork, so as soon as the child begins running, switch_to executes ret_from_fork, which proceeds directly to RESTORE_ALL and then returns to user space.

For kernel threads, there is no concept of a "child process return." The newly created kernel thread simply runs and exits when done. This is because its execution strategy is already defined at creation time, eliminating the need to return to a common point and use return values to distinguish parent from child. However, kernel threads are indeed created using the same mechanism as user processes — the child is still duplicated from the parent in copy_process, with no difference in that aspect. So how can it avoid returning to the origin?

Linux uses a trick: it fabricates a parent process context when creating a kernel thread:

int kernel_thread(int (*fn)(void *), void * arg, unsigned long flags)
{
        struct pt_regs regs;                // Fabricated parent context, filled according to kernel mechanisms below
        memset(&regs, 0, sizeof(regs));
        regs.ebx = (unsigned long) fn;      // Execution behavior for the child (i.e., the kernel thread)
        regs.edx = (unsigned long) arg;     // Arguments
        ...
        regs.eip = (unsigned long) kernel_thread_helper;     // This function manages the child's execution and exit
        ...
        return do_fork(flags | CLONE_VM | CLONE_UNTRACED, 0, &regs, 0, NULL, NULL);  // Actually create the child process
}

__asm__(".section .text\n"
        ".align 4\n"
        "kernel_thread_helper:\n\t"   // This label function manages the kernel child process
        "movl %edx,%eax\n\t"          // Overwrites the eax set to 0 in copy_thread — shows eax is not kept 0 like in user process creation
        "pushl %edx\n\t"              // edx holds the kernel thread function's argument
        "call *%ebx\n\t"              // ebx holds the kernel thread function pointer
        "pushl %eax\n\t"              // Push the return value of the kernel thread function
        "call do_exit\n"              // Call do_exit with the function's return value as argument
        ".previous");

There is a reason why, when creating a kernel thread, the fabricated regs.eip is set to kernel_thread_helper instead of directly to the target function. Overall, kernel_thread_helper provides the kernel child process with a complete execution environment and lifecycle, including proper exit handling. If the target function were called directly, it would have to handle its own exit. Process creation and termination are part of the execution mechanism, which should not be the responsibility of the creator — the creator should only define the policy. The mechanism must be provided by the kernel framework.

Additionally, note the CLONE_VM flag in do_fork. Do kernel threads have an address space? Actually, they do not. This flag is set purely for efficiency. Examining the code shows that Linux avoids switching the cr3 register when switching task_struct for tasks sharing VM (on x86). Since kernel threads lack an mm_struct, they use an active_mm field to borrow the mm of the previous process. All kernel-space memory mappings are identical across processes, and kernel threads only use kernel-space mappings. Thus, cr3 switching is avoided. The processor then enters lazy mode: only when the borrowed process's TLB is flushed is cr3 switched to the physical address of swapper_pg_dir. In fact, swapper_pg_dir is the page directory that kernel threads should naturally use. In this sense, all kernel threads can be viewed as belonging to a single kernel process — the one using swapper_pg_dir as its page directory. Indeed, a process is essentially a thread of execution with its own page directory. For efficiency, kernel threads borrow the mm_struct of user-space processes. Apart from swapper_pg_dir, which is the standard page directory for kernel threads, no new PGD should be allocated. Hence, the CLONE_VM flag ensures no mm_struct (and thus no PGD) is allocated, avoiding duplication. The issue of sharing the parent's mm can be resolved by releasing the old mm and switching to init_mm. Furthermore, as discussed next, because kernel memory mappings are identical, borrowing any process's mm makes kernel operations more efficient.

The TLB lazy mode is described in my article "Lazy Mode of TLB Flush." Briefly, on a single CPU, TLB flush is an active process and thus straightforward — active processes usually have deterministic behavior. However, on SMP systems, it becomes more complex. Consider the following:

static inline task_t * context_switch(runqueue_t *rq, task_t *prev, task_t *next)
{
        struct mm_struct *mm = next->mm;
        struct mm_struct *oldmm = prev->active_mm;
        if (unlikely(!mm)) {             // Kernel thread
                next->active_mm = oldmm;
                atomic_inc(&oldmm->mm_count);
                enter_lazy_tlb(oldmm, next);  // Does nothing on uniprocessor
        } else
                switch_mm(oldmm, mm, next);   // Perform switch
        if (unlikely(!prev->mm)) {
                prev->active_mm = NULL;
                WARN_ON(rq->prev_mm);
                rq->prev_mm = oldmm;
        }
        ...
}

static inline void enter_lazy_tlb(struct mm_struct *mm, struct task_struct *tsk)
{
#ifdef CONFIG_SMP
        unsigned cpu = smp_processor_id();
        if (per_cpu(cpu_tlbstate, cpu).state == TLBSTATE_OK)
                per_cpu(cpu_tlbstate, cpu).state = TLBSTATE_LAZY;   // Set this CPU's cpu_tlbstate to lazy mode
#endif
}

On SMP systems, flushing the TLB requires sending inter-processor interrupts (IPIs) to all processors. When a CPU in lazy mode receives a TLB-flush IPI, it removes itself from the `cpu_vm_mask` of `active_mm` in `cpu_tlbstate`, indicating that it should no longer receive TLB-flush IPIs. This is because CPUs in lazy mode are currently running kernel threads, and all processes share identical kernel-space mappings — so using any `mm` is acceptable. However, this is not entirely safe. For example, if a kernel thread is using a borrowed `mm` and the original process (and its `mm`) is freed on another CPU, even though `atomic_inc(&oldmm->mm_count)` delays the release, it's still suboptimal. Why use someone else's `mm` when `swapper_pg_dir` — which is always safe and never freed — is available? Using borrowed `mm` only consumes memory. Therefore, when a CPU in lazy mode receives its first TLB-flush IPI, it should load its page directory with a safe value (i.e., `swapper_pg_dir`) before declaring it will no longer accept TLB-flush IPIs. Remember: once a non-kernel thread begins execution, the CPU must resume accepting TLB-flush IPIs — that is, exit lazy mode.

This article was translated from Chinese to English with AI assistance and a light human review. The original is published at Sienovo Blog. The original Chinese source is at CSDN. Learn more about Sienovo edge AI computing.

How to Install MySQL 5.0 on Linux

Leo Liu — Thu, 28 May 2026 07:17:02 +0000

Originally published on the Sienovo Engineering Blog. Sienovo is the overseas brand of 深圳信迈 (Shenzhen Xinmai), building edge AI computing solutions for industrial video analytics.

For the clearest guide, refer to Baidu Wenku: http://wenku.baidu.com/view/571968976bec0975f465e25b.html

=============================================

(I) How to Install MySQL Database on Linux

=============================================

My first setback was with RPM installation. It reported a MySQL version conflict, preventing the installation from proceeding. After querying with rpm -q, I attempted to remove the system's pre-installed version (3.x.x) using rpm -e. However, RPM mercilessly returned a slew of other errors. Due to dependencies with other software, version 3.x.x could not be removed. My initial positive impression of RPM vanished in an instant.

As the saying goes, "Among three people, there must be one who can be my teacher." I immediately found Alin on MSN. When I asked him how to uninstall or upgrade MySQL using RPM, he gleefully told me that he upgraded it in just one minute using "Debian." He then added that he "doesn't know how" to use RPM, all while smirking. I knew he was getting back at me; last time he kept recommending "Debian" and I ignored him, so he found his chance. Actually, I do think "Debian" is good, it's just that the name sounds a bit unpleasant...

Since RPM wasn't working, I had to settle for a different approach. This time, I chose the binary installation package. I found an installation guide from a pioneer via Google as a reference. I quickly followed in their footsteps, only to be bounced back, bruised and battered. The previous experience didn't suit me; it seems there truly is no silver bullet in this world.

It became clear that impatience wouldn't solve the problem. I decided to calm down and found several more articles introducing MySQL installation. After multiple attempts, I finally found a small path through the thorny thicket...

Download the binary MySQL installation package from http://www.mysql.com // This MySQL is a binary version, no compilation needed
# chmod 755 mysql-standard-5.0.15-linux-gnu-i686-glibc23.tar.gz //
# tar xfz mysql-standard-5.0.15-linux-gnu-i686-glibc23.tar.gz // Copy the extracted directory to /usr/local/ and rename it to mysql
# groupadd mysql # useradd mysql -g mysql // Create the mysql group // Create the mysql user and add them to the mysql group
# cp /usr/local/mysql/support-files/my-medium.cnf /etc/my.cnf // In the support-files directory, there are 4 template files. We choose one of them as the MySQL configuration file, overwriting /etc/my.cnf (the system's default configuration, which sets performance parameters and some MySQL path parameters).
# cd /usr/local/mysql # ./scripts/mysql_install_db --user=mysql // Enter the mysql directory // Initialize tables and specify that the mysql user will access them. After initializing tables, set access permissions for the mysql and root users.
# chown -R root . // Set root to have access to /usr/local/mysql
# chown -R mysql data // Set the mysql user to have access to /usr/local/mysql/data, which contains MySQL database files. This directory is configured in /etc/my.cnf and created during mysql_install_db.
# chown -R mysql data/. // Set the mysql user to have access to all files under /usr/local/mysql/data/mysql
# chgrp -R mysql . // Set the mysql group to have access to /usr/local/mysql
# /usr/local/mysql/bin/mysqld_safe --user=mysql & // Run MySQL // If there are no issues, you should see a prompt similar to this: [1] 42264 # Starting mysqld daemon with databases from /usr/local/mysql/var // If a statement like "mysql ended" appears, it means MySQL did not start correctly. You can check the log for issues; the log file is usually configured in /etc/my.cnf. Most problems are caused by incorrect permission settings.
// Use the following command to change the MySQL password # /usr/local/mysql/bin/mysqladmin -u root password yourpassword // The default installation has no password; for security, you must change it immediately.
# cp support-files/mysql.server /etc/rc.d/init.d/mysqld # chmod 700 /etc/init.d/mysqld # chkconfig --add mysqld # chkconfig --level 345 mysqld on // Copy a script from the compilation directory // Configure MySQL to start automatically on every boot.
# service mysqld start # netstat -atln // Start the mysqld service // Check if port 3306 is open. Remember to open this port in the firewall.

============================================

(II) How to Install MySQL Database on Linux

============================================

Using Options on the Command Line

Program options specified on the command line follow these rules:
· Options immediately follow the command name.
· Option parameters begin with one or two hyphens, depending on whether they have a short or long name. Many options have both forms. For example, -? and --help are the short and long names for the option that tells a MySQL program to display help messages.
· Option names are case-sensitive. -v and -V are both valid but have different meanings. (They are the short names for the --verbose and --version options).
· Some options are followed by an option value. For example, -h localhost or --host=localhost indicates the MySQL server host for the client program. The option value tells the program the hostname where the MySQL server is running.
· For long options with values, separate the option name and value with an '='. For short options with values, the option value can immediately follow the option letter, or there can be a space between them. (-hlocalhost and -h localhost are equivalent). An exception to this rule is the option for specifying the MySQL password. This option can be in the form --password=pass_val or --password. In the latter case (no password value given), the program will prompt for the password. The password option can also be given in the short form -ppass_val or -p. However, for the short form, if a password value is given, it must immediately follow the option without any intervening space. The reason for this requirement is that if there is a space after the option, the program has no way of knowing whether the subsequent argument is the password value or some other argument. Therefore, the following two commands have completely different meanings:
[pre]· shell> mysql -ptest[/pre][pre]· shell> mysql -p test[/pre]The first command tells mysql to use the password 'test' but does not specify a default database. The second command tells mysql to prompt for the password and use 'test' as the default database.

I. Introduction

I have wanted to use Linux for a long time, but without a specific task, I never systematically learned it. Recently, due to work requirements, I had to use MySQL on Linux. I initially thought that with my experience using SQL Server on Windows, installing MySQL on Linux would be a breeze. However, I encountered many detours and problems during the actual installation and use of MySQL, as Linux and Windows are fundamentally very different. To help beginners like me avoid detours and get started quickly, I wrote this article, hoping it will be helpful to you.

II. Installing MySQL

Download MySQL Installation Files
To install MySQL, you need the following two files:
MySQL-server-5.0.9-0.i386.rpm
MySQL-client-5.0.9-0.i386.rpm
Download them from: http://dev.mysql.com/downloads/mysql/5.0.html. Open this page, scroll down to "Linux x86 RPM downloads," find "Server" and "Client programs," and download the two required RPM files.
Install MySQL
RPM files are software packages developed by Red Hat. RPM allows Linux to install software packages without many complex procedures. Commonly used parameters for the rpm command during installation are –ivh, where i indicates installing the specified RPM package, v indicates verbose information during installation, and h indicates displaying '#' symbols during installation to show the current installation progress. This symbol will continue until the installation is complete.
1) Install Server
Run the following command in the directory containing the two RPM files:
[root@test1 local]# rpm -ivh MySQL-server-5.0.9-0.i386.rpm
The following information will be displayed:
warning: MySQL-server-5.0.9-0.i386.rpm: V3 DSA signature: NOKEY, key ID 5072e1f5
Preparing... ########################################### [100%]
1:MySQL-server ########################################### [100%]
...... (display omitted)
/usr/bin/mysqladmin -uroot password 'new-password'
/usr/bin/mysqladmin -uroot -h test1 password 'new-password'
...... (display omitted)
Starting mysqld daemon with databases from /var/lib/mysql

If the above information appears, the server installation is complete. To test if it was successful, you can run netstat to see if the MySQL port is open. If it is open, the service has started, and the installation is successful. The default MySQL port is 3306.
[root@test1 local]# netstat -nat
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address Foreign Address State
tcp 0 0 0.0.0.0:3306 0.0.0.0:* LISTEN
The above display shows that the MySQL service has started.
2) Install Client
Run the following command:
[root@test1 local]# rpm -ivh MySQL-client-5.0.9-0.i386.rpm
warning: MySQL-client-5.0.9-0.i386.rpm: V3 DSA signature: NOKEY, key ID 5072e1f5
Preparing... ########################################### [100%]
1:MySQL-client ########################################### [100%]
The display indicates that the installation is complete.
Use the following command to connect to MySQL and test for success.

III. Logging into MySQL

The command to log into MySQL is mysql. The syntax for mysql is as follows:
mysql [-u username] [-h host] [-p[password]] [dbname]
username and password are the MySQL username and password, respectively. The initial MySQL administrative account is root with no password. Note: This root user is not the Linux system user. The default MySQL user is root. Since there is no initial password, you only need to type mysql for the first login.
[root@test1 local]# mysql
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 1 to server version: 4.0.16-standard
Type 'help;' or '\h' for help. Type '\c' to clear the buffer.
mysql>
If the "mysql>" prompt appears, congratulations, the installation was successful!
The login format after adding a password is as follows:
mysql -u root -p
Enter password: (enter password)
Here, -u is followed by the username, and -p prompts for the password. Press Enter and then enter the password at the prompt.

Note: This mysql file is in the /usr/bin directory and is not the same file as the startup file /etc/init.d/mysql which will be discussed later.

IV. Several Important MySQL Directories

After MySQL is installed, unlike SQL Server, it is not installed in a single directory. Its database files, configuration files, and command files are in different directories. Understanding these directories is very important, especially for Linux beginners, because the Linux directory structure itself is quite complex. If you don't understand the MySQL installation directories, you cannot talk about in-depth learning.

Below are descriptions of these directories.

Database Directory
/var/lib/mysql/
Configuration Files
/usr/share/mysql (mysql.server command and configuration files)
Related Commands
/usr/bin (mysqladmin, mysqldump, and other commands)
Startup Scripts
/etc/rc.d/init.d/ (directory for the MySQL startup script file)

V. Changing the Login Password

MySQL has no default password. The importance of adding a password after installation is self-evident.

Command
usr/bin/mysqladmin -u root password 'new-password'
Format: mysqladmin -u username -p old_password password new_password
Example
Example 1: Add a password '123456' for root.
Type the following command:
[root@test1 local]# /usr/bin/mysqladmin -u root password 123456
Note: Since root initially has no password, the -p old_password part can be omitted.
Test if the modification was successful
1) Log in without a password
[root@test1 local]# mysql
ERROR 1045: Access denied for user: 'root@localhost' (Using password: NO)
An error is displayed, indicating that the password has been changed.
2) Log in with the modified password
[root@test1 local]# mysql -u root -p
Enter password: (enter the modified password 123456)
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 4 to server version: 4.0.16-standard
Type 'help;' or '\h' for help. Type '\c' to clear the buffer.
mysql>
Success!
This changes the password using the mysqladmin command. You can also change the password by modifying the database.

VI. Starting and Stopping

Start
After MySQL is installed, the startup file mysql is in the /etc/init.d directory. To start it, run the following command:
[root@test1 init.d]# /etc/init.d/mysql start
Stop
/usr/bin/mysqladmin -u root -p shutdown
Automatic Startup
1) Check if MySQL is in the automatic startup list
[root@test1 local]# /sbin/chkconfig --list
2) Add MySQL to your system's startup service group
[root@test1 local]# /sbin/chkconfig – add mysql
3) Remove MySQL from the startup service group.
[root@test1 local]# /sbin/chkconfig – del mysql
1. The command to restart the MySQL service is service mysql restart or /etc/init.d/mysql restart.

VII. Changing MySQL Directory

The default data file storage directory for MySQL is /var/lib/mysql. If you want to move the directory to /home/data, you need to perform the following steps:

Create the data directory in /home
cd /home
mkdir data
Stop the MySQL service process:
mysqladmin -u root -p shutdown
Move the entire /var/lib/mysql directory to /home/data
mv /var/lib/mysql /home/data/
This moves the MySQL data files to /home/data/mysql.
Find the my.cnf configuration file
If there is no my.cnf configuration file in the /etc/ directory, find a *.cnf file in /usr/share/mysql/, copy one to /etc/, and rename it to my.cnf. The command is as follows:
[root@test1 mysql]# cp /usr/share/mysql/my-medium.cnf /etc/my.cnf
Edit MySQL's configuration file /etc/my.cnf
To ensure MySQL works correctly, you need to specify the location where the mysql.sock file is generated. Modify the socket=/var/lib/mysql/mysql.sock line by changing the value to the right of the equals sign to: /home/mysql/mysql.sock. Perform the following operation:
vi my.cnf (Use the vi tool to edit the my.cnf file, find the following data, and modify it)
# The MySQL server
[mysqld]
port = 3306
#socket = /var/lib/mysql/mysql.sock (Original content, commented out with "#" for safety)
socket = /home/data/mysql/mysql.sock (Add this line)
Modify the MySQL startup script /etc/rc.d/init.d/mysql
Finally, you need to modify the MySQL startup script /etc/rc.d/init.d/mysql, changing the datadir=/var/lib/mysql line by replacing the path to the right of the equals sign with your current actual storage path: home/data/mysql.
[root@test1 etc]# vi /etc/rc.d/init.d/mysql
#datadir=/var/lib/mysql (Comment out this line)
datadir=/home/data/mysql (Add this line)
Restart the MySQL service
/etc/rc.d/init.d/mysql start
Or restart Linux with the reboot command.
If it works correctly, the move was successful. Otherwise, recheck the previous 7 steps.

VIII. Common MySQL Operations

Note: Every command in MySQL must end with a semicolon (;).

Show Databases
mysql> show databases;
+----------+
| Database |
+----------+
| mysql |
| test |
+----------+
2 rows in set (0.04 sec)
After MySQL is installed, there are two databases: mysql and test. The mysql database is very important; it contains MySQL's system information. When we change passwords and add new users, we are actually operating on the relevant tables in this database.
Show Tables in a Database
mysql> use mysql; (Open the database; to operate on a database, you must open it, similar to FoxPro)
Database changed

mysql> show tables;
+-----------------+
| Tables_in_mysql |
+-----------------+
| columns_priv |
| db |
| func |
| host |
| tables_priv |
| user |
+-----------------+
6 rows in set (0.01 sec)

Show Table Structure:
describe table_name;
Show Records in a Table:
select * from table_name;
For example: Show records in the user table in the mysql database. All users who can operate on MySQL users are in this table.
Select * from user;
Create Database:

`

FPGA Applications in Healthcare, Illustrated by 4K Medical Endoscopy

Leo Liu — Wed, 27 May 2026 07:23:06 +0000

Originally published on the Sienovo Engineering Blog. Sienovo is the overseas brand of 深圳信迈 (Shenzhen Xinmai), building edge AI computing solutions for industrial video analytics.

Introduction

With technological advancements, medical imaging—one of the major achievements of modern science—has found diverse applications in non-invasive diagnosis and treatment. One such application is endoscopy, which became more widespread in the 1990s when charge-coupled devices made it possible to transmit images to displays. To assist doctors in better identifying and locating lesions, manufacturers have continuously improved endoscope resolution, progressing from 1080P to today’s 4K for human medical endoscopes. In addition, innovative technologies such as fluorescence and 3D imaging have been integrated to further aid physicians in accurate diagnosis and surgical procedures. Recently, China's Ministry of Industry and Information Technology released the Medical Equipment Industry Development Plan (2021–2025), which outlines detailed strategic directions for key areas of medical equipment development, including the strategic goal of breaking through in key technologies for medical endoscopes and other diagnostic imaging devices.

Technical Challenges

In clinical settings, the maximum allowable image latency for endoscopic applications ranges between 50 and 150 milliseconds. However, during surgery, endoscopes must respond in real time or near real time while simultaneously performing image correction, color noise reduction, edge enhancement, zooming, and other functions. Additionally, the end device should be as compact as possible, support 4K resolution, 3D imaging, fluorescence, and be compatible with SDI/HDMI interfaces. While 4K requires only one high-definition camera, fluorescence and 3D each require an additional HD camera, posing significant challenges to core board resources, data transmission speed, processing throughput, and algorithmic efficiency.

Solution

An endoscopic system built around FPGA SoC technology can deliver real-time 4K video streaming. A 4K image sensor captures visual data, while image signal processing is handled by the Mercury+ XU8 FPGA (SoC) module. The captured video stream is fed into the Mercury+ XU8 FPGA (SoC) module for preprocessing, then passed through an image management and storage unit before being displayed on a high-resolution monitor via a display interface for the surgeon’s use. The Mercury+ XU8 core module—long featured on Xilinx’s homepage—comes in three models: XCZU4CG, XCZU5EV, and XCZU7EV. Users can easily upgrade to higher-end models when more resources are needed, enabling simple and seamless system evolution.

Mercury+ XU8 Core Module

Block Diagram

Besides the Mercury+ XU8 FPGA (SoC), other SoCs such as Mercury+ XU5 and Mercury+ XU9 can also be considered for this application. Using the Mercury+ XU8 FPGA (SoC) module enables high integration of the hardware system, significantly shortening development time. Additionally, support for various peripheral interfaces allows faster and more convenient future functionality updates and expansions. Thanks to Ruishu Yingke’s extensive product lineup, users can choose from multiple core module options across Xilinx Kintex-7, Zynq-7000, and Zynq Ultrascale+ MPSoC families. These core modules are pin-compatible within their series (Mars, Mercury, Andromeda), meaning users can plan clear upgrade paths, greatly reducing engineering effort during upgrades—even allowing last-minute core module changes during project development. FPGA core modules have a minimum expected lifecycle of over 10 years, with hardware designs emphasizing long-term availability and performance, ensuring all products remain deliverable for extended periods.

Core Module Series

FPGA (Field-Programmable Gate Array), a key innovation by Xilinx, is renowned for its programmability and flexibility. Initially, FPGAs were used primarily to simulate ASICs before mask processing and mass production. However, ASICs require high customization and become cost-prohibitive at low volumes, so FPGAs and ASICs have coexisted without conflict, each serving distinct roles. Later, with the emergence of accelerators and increased computational demands, FPGAs evolved into parallel computing devices rivaling GPUs in prominence.

Today, FPGAs have entered data center applications, where they offer reduced component count and superior power efficiency compared to CPUs and GPUs. Backed by Xilinx’s three strategic pillars—“Data Center First,” “Accelerating Core Markets,” and “Driving Adaptive Computing”—its ACAP platform and Alveo accelerator cards have become highly competitive in the data center market.

Beyond that, Xilinx has demonstrated its “integrated SmartNIC platform” for cloud service providers, “FPGA TCON” solutions for consumer electronics, and Zynq SoC series solutions for industrial applications.

In fact, according to Xilinx, the healthcare sector now represents a significant portion of its revenue and has been growing steadily at 11–15% annually. What FPGA products enable Xilinx to capture this market, and what role do FPGA devices play in medical equipment?

Recently, Xilinx presented its latest achievements in medical science and medical devices to journalists; reporters from 21ic China Electronics Network attended the briefing.

Where Can FPGA Devices Be Used?

Data shows that global per capita healthcare spending continues to rise. With an aging population, consumers have high expectations regarding both medical quality and cost. Meanwhile, the pandemic has heightened demand for early disease detection and rapid diagnostic analysis, necessitating lower medical device costs and higher computational power.

FPGA devices offer inherent programmability, allowing developers to avoid the high non-recurring engineering (NRE) costs associated with ASICs, eliminate minimum order quantities, and reduce risks and losses from multi-chip iterations. The medical industry is one of the most closely linked to technological advancement, and as FPGA technology continuously evolves, new devices emerge, driving changes in treatment methods, pathways, and philosophies.

Subh Bhattacharya, Global Business Marketing Lead for Medical Sciences at Xilinx

According to Subh Bhattacharya, Xilinx FPGA applications in healthcare fall into three main categories: clinical applications, medical imaging, and diagnostic analytics.

Clinical Environments

Clinical equipment varies widely in type and quantity, requiring highly flexible solutions—making FPGAs ideal. Notably, some devices directly impact patient safety and thus demand extremely fast boot times, high reliability, and low latency; others prioritize portability, requiring low power consumption and small form factors.

As explained by Subh, Xilinx’s Zynq UltraScale+ MPSoC (hereafter referred to as “ZU+ MPSoC”) is a highly integrated platform combining multiple processors with programmable logic, as well as integrated information security and functional safety features. Subh emphasized that this platform’s powerful capabilities and performance make it well-suited for clinical applications ranging from cloud to edge.

Subh presented several examples of how this platform addresses clinical needs:

First, a medical AI solution developed in collaboration with Spline.AI and AWS (Amazon Web Services), using the ZU+ MPSoC ZCU104 platform as an edge device to implement a high-accuracy, low-latency deep learning model and reference design for medical X-ray classification. This solution can autonomously predict diseases from chest X-rays, including COVID-19 and pneumonia, and supports custom model development for clinical use. Additionally, the ZCU104 supports development using the open-source PYNQ framework and can be extended and deployed via AWS IoT Greengrass. This solution leverages the ZU+ MPSoC’s high performance and scalability to deliver precise diagnostics at low cost.

Second, Xilinx supports Olympus in its core endoscope technology, leveraging the ZU+ MPSoC’s fast startup, low power, and low-latency characteristics.

Third, Xilinx powers Clarius’s ultra-portable, high-performance ultrasound system, utilizing the ZU+ MPSoC’s dual on-chip ARM processors and FPGA in a compact package to achieve extreme portability.

Historically, Zynq SoC was Xilinx’s first product integrating ARM cores, launched in 2011. At the time, it was called an “extensible processing platform,” aimed at expanding into embedded applications. Previously, FPGAs were mostly used as auxiliary chips; after integrating more functionalities into a single SoC platform, components such as ARM GPUs, data security processors, and functional safety processors were consolidated onto one chip. Subh noted that this transformation boosted Xilinx’s annual revenue growth from 5–6% to 14–15%, a 2.5x increase attributed entirely to this technological platform.

Additionally, Subh demonstrated ZU+ MPSoC’s medical security solutions. “Globally, over 100 million medical IoT devices are currently deployed, projected to grow to 161 million by 2020. Healthcare executives identify 59% privacy concerns, 55% legacy system integration, and 54% security issues as the top three barriers to IoT adoption in healthcare.”

Subh explained that Xilinx leverages its programmable platform to continuously adapt to new security measures—both software and hardware updates. These capabilities are implemented in the SoC as certified and encrypted boot, secure boot, measured boot, secure application communication, and cloud-based monitoring.

Medical Imaging

The use of FPGA devices in large-scale medical imaging equipment is now standard practice. As Subh explained, major imaging modalities include CT, ultrasound, X-ray, PET, and MRI scanners.

For medical imaging, Zynq UltraScale+ MPSoC remains applicable. Beyond that, Subh highlighted the Versal ACAP series—considered the next-generation MPSoC—which holds significant advantages in imaging applications.

Versal ACAP integrates ARM multi-core processors, programmable logic, and DSP blocks, and adds AI Engines—units based on SIMD and VLIW architectures—capable of parallel processing for similar operations.

Subh showcased a solution for ultrasound image reconstruction and computer-aided diagnosis. With Xilinx’s hardware and software support, this solution reduces power consumption and thermal footprint, lowers overall system cost, extends equipment lifespan, and enables low-latency edge inference. Despite market complexity, Xilinx’s technology significantly enhances productivity.

Diagnostic Analytics

Subh noted that beyond SoCs and FPGAs, Xilinx offers plug-and-play Alveo accelerator cards. As PCIe-based solutions, these cards greatly reduce development time. Alveo cards work with any standard PC, accelerating both general CPU tasks and GPU workloads, delivering high throughput and ultra-low latency. Their unique computational power and adaptability significantly speed up various medical applications.

United Imaging, a Chinese company, found that replacing traditional GPUs with Alveo U200 accelerator cards resulted in lower technology costs and power consumption, without sacrificing performance or development timelines.

FPGA vs. CPU & GPU

Solutions using CPUs or GPUs in medical devices are common. Why do FPGAs perform so exceptionally well—possessing an almost “magical” ability to replace CPUs and GPUs? In reality, both CPUs and GPUs follow the von Neumann architecture, whereas FPGAs break through architectural limitations, achieving superior energy efficiency.

Specifically, CPUs and GPUs rely on SIMD (Single Instruction, Multiple Data) to execute operations involving memory, decoders, arithmetic units, and branch logic. In contrast, FPGAs define the function of each logic unit during programming, eliminating the need for instruction decoding. Moreover, CPUs and GPUs share memory, requiring arbitration and maintaining cache coherency across execution units. FPGAs, however, define communication requirements during configuration and do not require shared memory for inter-unit communication.

As a result, FPGAs deliver exceptional floating-point multiplication performance with lower latency than GPUs—even in floating-point operations—because FPGAs support both pipelined and data-level parallelism, while GPUs only support data parallelism.

In terms of computational power, Xilinx has evolved FPGA devices into SoCs for acceleration and adaptability. Scalar engines—including ARM, application processors, and real-time processors—handle acceleration, while the adaptive engine is based on programmable logic (FPGA), supplemented by intelligent engines such as DSPs. On the Versal ACAP platform, AI Engines further enhance acceleration and adaptability.

“In medical applications like endoscopy, patients undergoing surgery share a critical requirement: extremely low latency, often requiring real-time processing. From image capture by the camera, through the processing pipeline, to display output, the entire process may take less than 20 microseconds. CPUs and GPUs cannot achieve such low latency—this is FPGA’s greatest advantage over CPUs and GPUs,” Subh continued. “Additionally, in terms of power, cost, and integration, Xilinx SoC-based FPGAs offer superior benefits.”

“In many fields like visualization, GPUs have been used for years. FPGAs are certainly capable, but we focus on our strengths—efficient data movement within closed systems rather than intermittent memory uploads,” Subh admitted.

Analyzing FPGA Applications in Healthcare Across Layers

Xilinx stands alone in the medical field by simultaneously delivering industry-leading AI latency and performance, extended lifecycle, high quality, reliability, security, and real-time deterministic control and interfacing.

Beyond providing FPGA and SoC hardware platforms, Xilinx has developed the Vitis AI unified software platform specifically to lower FPGA development barriers and meet broad market application needs. This software platform has been previously covered extensively; it enables algorithm engineers without hardware design experience to directly implement their algorithms.

Xilinx’s medical solutions have helped Illumina perform genomic analysis on critically ill newborns, accelerated ICU and critical care patient communication via Eyetech’s eye-tracking tablet, and partnered with Mindray to combat the pandemic. In subtle yet profound ways, FPGAs add a layer of reverence for life.

In my view, Xilinx’s FPGA devices derive their market competitiveness from two key strengths: high-performance acceleration and adaptability. On one hand, FPGAs, ARM processors, application processors, real-time processors, DSPs, and AI engines are highly integrated via SoC and software, enhancing both computational power and application scalability. On the other hand, the inherent low latency of FPGAs makes them a natural fit for medical applications with stringent timing requirements.

From a market perspective, the pandemic has driven sustained growth in demand for medical devices, particularly those requiring large-scale data analysis and high portability—both aligning perfectly with FPGA SoC capabilities. Meanwhile, rising medical standards and market consolidation (Matthew effect) have increased demand for energy-efficient, low-power FPGA solutions.

From a software standpoint, Xilinx’s Vitis platform serves diverse user groups: hardware engineers fluent in HDL, software engineers skilled in mainstream programming languages, and algorithm engineers experienced with TensorFlow, Caffe, and PyTorch. This flexibility empowers innovative startups to bring creative ideas to life.

As illustrated by Xilinx’s presentation, FPGAs have a place in both large-scale and portable medical devices.

How will Xilinx’s path of medical innovation evolve in the future? Subh stated that Xilinx will continue to increase integration and reduce package size in medical products, while advancing heterogeneous computing to improve efficiency and performance.

Demystifying C Language Pointers

Leo Liu — Tue, 26 May 2026 07:09:05 +0000

Originally published on the Sienovo Engineering Blog. Sienovo is the overseas brand of 深圳信迈 (Shenzhen Xinmai), building edge AI computing solutions for industrial video analytics.

A pointer is a special variable whose stored value is interpreted as an address in memory. To fully understand a pointer, you need to clarify four aspects: the pointer's type, the type it points to, its value (or the memory region it points to), and the memory space occupied by the pointer itself. Let's go through each in detail.

First, declare a few pointers as examples:

Example One:
(1) int* ptr;

(2) char* ptr;

(3) int** ptr;

(4) int(*ptr)[3];

(5) int*(*ptr)[4];

Pointer Type

From a syntactic perspective, if you remove the pointer name from its declaration, what remains is the pointer's type — the inherent type of the pointer itself. Let's examine the pointer types in Example One:
(1) int* ptr; // pointer type is int*

(2) char* ptr; // pointer type is char*

(3) int** ptr; // pointer type is int**

(4) int(*ptr)[3]; // pointer type is int(*)[3]

(5) int*(*ptr)[4]; // pointer type is int*(*)[4]

Pretty straightforward, right? The method for identifying a pointer's type is quite simple.

Type Pointed To by the Pointer

When accessing the memory region via a pointer, the type pointed to by the pointer determines how the compiler interprets the contents of that memory region.

Syntactically, remove the pointer name and the * operator to its left from the declaration — what remains is the type the pointer points to. For example:
(1) int* ptr; // type pointed to is int

(2) char* ptr; // type pointed to is char

(3) int** ptr; // type pointed to is int*

(4) int(*ptr)[3]; // type pointed to is int()[3]

(5) int*(*ptr)[4]; // type pointed to is int*()[4]

The type pointed to plays a critical role in pointer arithmetic.

The pointer's type (i.e., the type of the pointer itself) and the type it points to are two distinct concepts. As you become more familiar with C, you'll realize that distinguishing between these two types is key to mastering pointers. I've read many books, and some poorly written ones conflate these two concepts, leading to contradictions and confusion.

Pointer Value, or the Memory Region (Address) It Points To

The pointer's value is the numerical value it stores, which the compiler interprets as a memory address rather than a regular integer. In a 32-bit program, all pointer values are 32-bit integers because all memory addresses are 32 bits long. The memory region pointed to by the pointer starts at the address represented by the pointer's value and spans a length of sizeof(the type pointed to by the pointer). Hence, saying "a pointer's value is XX" is equivalent to saying "the pointer points to a memory region starting at address XX"; similarly, saying "a pointer points to a certain memory region" means "the pointer's value is the starting address of that region."

The memory region pointed to and the type pointed to are entirely different concepts. In Example One, the type pointed to is defined, but since the pointer hasn't been initialized, the memory region it points to is either nonexistent or meaningless.

Whenever you encounter a pointer, always ask: What is its type? What type does it point to? Where does it point?

Memory Space Occupied by the Pointer Itself

How much memory does a pointer occupy? You can determine this using sizeof(pointer type). On a 32-bit platform, a pointer occupies 4 bytes.

This concept is particularly useful when determining whether a pointer expression is an lvalue.

Pointer Arithmetic

A pointer can be incremented or decremented by an integer. This arithmetic differs significantly from regular integer arithmetic. For example:

Example Two:

char a[20];
int* ptr = a; ... ...
ptr++;

In this example, ptr is of type int*, pointing to int, and is initialized to point to array a. In line 3, ptr is incremented by 1. The compiler handles this by adding sizeof(int) — which is 4 in a 32-bit program — to the pointer's value. Since addresses are byte-addressed, ptr now points to an address 4 bytes higher than before.

Since a char is 1 byte, ptr initially pointed to the first 4 bytes starting at a[0], and now points to the 4 bytes starting at a[4].

We can use a pointer and a loop to traverse an array. For example:

Example Three:
int array[20];

int* ptr = array;

...

// Code to initialize array values omitted

...

for (i = 0; i < 20; i++)

{

　(*ptr)++;

　ptr++;

}

This example increments each element of the integer array by 1. Since ptr is incremented in each iteration, it accesses the next array element each time.

Another example:

Example Four:

char a[20];
int* ptr = a; ... ...
ptr += 5;

Here, ptr is increased by 5. The compiler computes this as ptr's value plus 5 * sizeof(int) — 5 * 4 = 20 in a 32-bit program. Since addresses are in bytes, ptr now points 20 bytes higher. Before the addition, ptr pointed to the first 4 bytes of a; afterward, it points outside the valid range of array a. While this may cause issues in practice, it is syntactically valid — demonstrating pointer flexibility.

If ptr were instead decreased by 5, the process would be similar, but ptr's value would be reduced by 5 * sizeof(int), moving it 20 bytes lower in memory.

In summary:

When a pointer ptrold is incremented by an integer n, the result is a new pointer ptrnew. ptrnew has the same type and points to the same type as ptrold. The value of ptrnew is ptrold's value plus n * sizeof(type pointed to by ptrold) bytes. Thus, ptrnew points to a memory region n * sizeof(type) bytes higher than ptrold.

Similarly, when ptrold is decremented by n, ptrnew has the same type and points to the same type, but its value is reduced by n * sizeof(type), pointing n * sizeof(type) bytes lower.

Operators & and *

Here, & is the address-of operator, and * is the "indirection operator" as referred to in textbooks.

The result of &a is a pointer. Its type is the type of a with an added *, the type it points to is the type of a, and the address it points to is the address of a.

The result of *p varies. In general, *p evaluates to the object pointed to by p — its type is the type pointed to by p, and its address is the address p points to.

Example Five:
int a = 12;

int b;

int* p;

int** ptr;

p = &a;

// &a results in a pointer of type int*, pointing to type int, at address of a.

*p = 24;

// *p here has type int and refers to the address p points to — clearly, *p is variable a.

ptr = &p;

// &p results in a pointer of type int** (p's type with an added *), pointing to type int*, at p's own address.

*ptr = &b;

// *ptr is a pointer, and &b is also a pointer — both have the same type and pointed-to type, so assigning &b to *ptr is valid.

**ptr = 34;

// *ptr refers to what ptr points to — here, a pointer. Applying * again yields an int variable.

Pointer Expressions

An expression whose final result is a pointer is called a pointer expression.

Examples:
Example Six:
int a, b;

int array[10];

int* pa;

pa = &a; // &a is a pointer expression.

int** ptr = &pa; // &pa is also a pointer expression.

*ptr = &b; // both *ptr and &b are pointer expressions.

pa = array;

pa++; // this is also a pointer expression.

Example Seven:
char* arr[20];

char** parr = arr; // if arr is treated as a pointer, it's a pointer expression

char* str;

str = *parr; // *parr is a pointer expression

str = *(parr + 1); // *(parr + 1) is a pointer expression

str = *(parr + 2); // *(parr + 2) is a pointer expression

Since a pointer expression results in a pointer, it possesses the same four attributes: pointer type, type pointed to, memory region pointed to, and memory space occupied.

When the resulting pointer of a pointer expression clearly occupies its own memory, the expression is an lvalue; otherwise, it is not.

In Example Seven, &a is not an lvalue because it doesn't occupy definite memory. *ptr is an lvalue because it already occupies memory — *ptr is effectively pointer pa, and since pa has a defined location in memory, *ptr does too.

Relationship Between Arrays and Pointers

An array name can essentially be viewed as a pointer. For example:

Example Eight:
int array[10] = {0,1,2,3,4,5,6,7,8,9}, value;

...

...

value = array[0]; // equivalent to: value = *array;

value = array[3]; // equivalent to: value = *(array + 3);

value = array[4]; // equivalent to: value = *(array + 4);

Typically, the array name array represents the entire array, with type int[10]. But if treated as a pointer, it points to the 0th element, has type int*, and points to type int. Thus, *array equals 0 is unsurprising. Similarly, array + 3 is a pointer to the 3rd element, so *(array + 3) equals 3, and so on.

Example Nine:
char* str[3] = {

　"Hello, this is a sample! ",

　"Hi, good morning. ",

　"Hello world "

};

char s[80];

strcpy(s, str[0]); // equivalent to: strcpy(s, *str);

strcpy(s, str[1]); // equivalent to: strcpy(s, *(str + 1));

strcpy(s, str[2]); // equivalent to: strcpy(s, *(str + 2));

Here, str is a 3-element array, each element being a pointer to a string. Treating str as a pointer, it points to the 0th element, has type char**, and points to type char*.

*str is a pointer of type char*, pointing to char, and pointing to the first character 'H' of "Hello, this is a sample! ".

str + 1 is also a pointer, of type char**, pointing to the 1st element.

*(str + 1) is a pointer of type char*, pointing to the first character 'H' of "Hi, good morning.", and so on.

To summarize array names: declaring TYPE array[n] gives array two meanings:

It represents the entire array, with type TYPE[n].
It is a pointer of type TYPE*, pointing to type TYPE (the array element type), pointing to the 0th element, and occupying its own memory space — distinct from the memory of the 0th element. Its value cannot be modified; expressions like array++ are invalid.

In different expressions, array can play different roles.

In sizeof(array), array represents the entire array, so sizeof returns the total size.

In *array, array acts as a pointer, so the result is the value of the 0th element. sizeof(*array) returns the size of one element.

In array + n (n = 0,1,2,...), array acts as a pointer, so array + n is a pointer of type TYPE*, pointing to the nth element. Thus, sizeof(array + n) returns the size of a pointer.

Example Ten:
int array[10];

int(*ptr)[10];

ptr = &array;

Here, ptr is a pointer of type int(*)[10], pointing to type int[10], initialized with the array's base address. In ptr = &array, array represents the entire array.

This section mentioned sizeof(). So, what does sizeof(pointer_name) measure — the size of the pointer type itself or the size of the type it points to? The answer is the former. For example:
int(*ptr)[10];

Then in a 32-bit program:
sizeof(int(*)[10]) == 4

sizeof(int[10]) == 40

sizeof(ptr) == 4

In fact, sizeof(object) always returns the size of the object's own type, not any other type.

Relationship Between Pointers and Structure Types

You can declare a pointer to a structure type.

Example Eleven:
struct MyStruct

{

　int a;

　int b;

　int c;

}

MyStruct ss = {20, 30, 40};

// Declares structure object ss, initialized with 20, 30, 40.

MyStruct* ptr = &ss;

// Declares a pointer to ss, of type MyStruct*, pointing to MyStruct.

int* pstr = (int*)&ss;

// Declares another pointer to ss, but with different type and pointed-to type than ptr.

How to access the three members of ss via ptr?

Answer:

ptr->a;

ptr->b;

ptr->c;

How to access them via pstr?

Answer:

*pstr; // accesses member a

*(pstr + 1); // accesses member b

*(pstr + 2); // accesses member c

Although I've tested this code in MSVC++ 6.0, using pstr this way is non-standard. To understand why, consider how pointers access array elements:

Example Twelve:
int array[3] = {35, 56, 37};

int* pa = array;

Accessing array elements via pa:
*pa; // accesses element 0

*(pa + 1); // accesses element 1

*(pa + 2); // accesses element 2

The syntax resembles the non-standard way of accessing structure members via pointers.

All C/C++ compilers store array elements in contiguous memory with no gaps. However, when storing structure members, some compilers may require alignment (e.g., word or double-word), inserting "padding bytes" between members, creating gaps.

Thus, even if *pstr accesses member a of ss, *(pstr + 1) may not access member b — it might access padding bytes between a and b. This demonstrates pointer flexibility. If your goal is to detect padding bytes, this is a decent method.

The correct way to access structure members via pointers is as shown with ptr in Example Twelve.

Relationship Between Pointers and Functions

You can declare a pointer to a function:
int fun1(char*, int);

int(*pfun1)(char*, int);

pfun1 = fun1;

....

....

int a = (*pfun1)("abcdefg", 7); // call function via function pointer.

You can also use pointers as function parameters and use pointer expressions as arguments.

Example Thirteen:
int fun(char*);

int a;

char str[] = "abcdefghijklmn ";

a = fun(str);

...

...

int fun(char* s)

{

int num = 0;

for(int i = 0; i < strlen(s); i++)

{

num += *s; s++;

}

return num;

}

This function fun calculates the sum of ASCII values of all characters in a string. As mentioned earlier, an array name is also a pointer. When str is passed to s, the value of str (its address) is copied to s. s and str point to the same location but occupy separate storage. Incrementing s inside the function does not affect str.

Pointer Type Casting

When initializing or assigning a pointer, the left side is a pointer, and the right side is a pointer expression. In most earlier examples, the pointer type and the expression type match, as do the types they point to.

Example Fourteen:

float f = 12.3;
float* fptr = &f;
int* p;

Suppose we want p to point to float f. Can we write:
p = &f;

No. p has type int*, pointing to int. &f has type float*, pointing to float. The types don't match. At least in MSVC++ 6.0, pointer assignments require matching types and pointed-to types. (Other compilers may vary.) To achieve this, we need a "cast":
p = (int*)&f;

To convert a pointer p to type TYPE* pointing to TYPE, use:
(TYPE*)p;

This cast creates a new pointer of type TYPE*, pointing to TYPE, at the same address. The original pointer p remains unchanged.

When a function uses a pointer parameter, type conversion also occurs during argument passing.

Example Fifteen:
void fun(char*);

int a = 125, b;

fun((char*)&a);

...

...

void fun(char* s)

{

char c;

c = *(s + 3); *(s + 3) = *(s + 0); *(s + 0) = c;

c = *(s + 2); *(s + 2) = *(s + 1); *(s + 1) = c;

}

Note: This is a 32-bit program — int is 4 bytes, char is 1 byte. The function fun reverses the byte order of an integer. Notice that in the function call, &a has type int*, pointing to int, while the parameter s has type char*, pointing to char. Thus, a conversion from int* to char* occurs. Imagine the compiler creates a temporary char* temp, assigns temp = (char*)&a, then passes temp to s. The result: s is of type char*, points to char, and points to the address of a.

We know a pointer's value is the address it points to — a 32-bit integer in 32-bit programs. Can we assign an integer directly as a pointer value?
unsigned int a;

TYPE* ptr; // TYPE could be int, char, struct, etc.

...

a = 20345686;

ptr = 20345686; // want ptr to point to address 20345686 (decimal)

ptr = a; // same goal

Compiling this shows both lines are invalid. Is there no way? Yes, there is:
unsigned int a;

TYPE* ptr;

...

a = some_value; // must represent a valid address

ptr = (TYPE*)a; // this works.

Strictly speaking, this (TYPE*) differs slightly from casting in pointer conversions. Here, (TYPE*) treats the unsigned integer a as a memory address. a must represent a valid address; otherwise, using ptr may cause illegal access.

Can we reverse this — extract a pointer's value as an integer, then assign that integer as an address to another pointer? Yes. Example:

Example Sixteen:
int a = 123, b;

int* ptr = &a;

char* str;

b = (int)ptr; // extract ptr's value as integer

str = (char*)b; // assign integer as address to str

Now we know: a pointer's value can be extracted as an integer, and an integer can be assigned as a pointer address.

Pointer Safety Issues

Consider this example:

Example Seventeen:
char s = 'a';

int* ptr;

ptr = (int*)&s;

*ptr = 1298;

ptr is an int* pointing to int, at the address of s. In a 32-bit program, s occupies 1 byte, int occupies 4. The last statement modifies not only s's byte but also the next 3 higher bytes. What are those bytes? Only the compiler knows — they might hold critical data or even executable code. Your careless pointer use could corrupt them, causing a crash.

Another example:

Example Eighteen:

char a;
int* ptr = &a; ... ...
ptr++;
*ptr = 115;

This compiles and runs. But after incrementing ptr in line 3, it points to memory adjacent to a. What's there? Unknown — possibly critical data or code. Line 4 writes to that region — a serious error. Always know exactly where your pointer points. When using pointers with arrays, ensure you don't exceed array bounds, or similar errors occur.

In pointer casting: ptr1 = (TYPE*)ptr2, if sizeof(ptr2's type) > sizeof(ptr1's type), accessing ptr2's memory via ptr1 is safe. If sizeof(ptr2's type) < sizeof(ptr1's type), it's unsafe. Think about Example Seventeen to understand why.

http://embedfans.com/C/2007181016375897.htm

Summary of Chinese Character Encoding and Programming: How ASCII, Internal Code, Zone-Position Code, GB Code, and Unicode Are Converted

Leo Liu — Mon, 25 May 2026 07:30:20 +0000

Originally published on the Sienovo Engineering Blog. Sienovo is the overseas brand of 深圳信迈 (Shenzhen Xinmai), building edge AI computing solutions for industrial video analytics.

I. How do ASCII, internal code, zone-position code, GB code, and Unicode convert among each other? What are the conversion formulas?

The process in Chinese character systems involves zone-position code, national standard code (GB code), and internal machine code. The conversion relationships are as follows:

Convert zone-position code (decimal) to zone-position code (hexadecimal):

Convert the zone number and position number separately into hexadecimal.

For example, if a character's zone-position code is 5448, convert 54 to hexadecimal 36, and 48 to hexadecimal 30. The resulting hexadecimal code is 3630.
GB Code = Zone-Position Code (hex) + 2020H

Example: 3630H + 2020H = 5050H → This yields the GB2312 national standard code.
Internal Machine Code = GB Code + 8080H

Example: 5050H + 8080H = D0D0H.

ASCII uses a single byte with the highest bit set to 0, which distinguishes it from Chinese character encoding.

Unicode is a character encoding standard proposed in Europe. If a large data packet contains double-byte values greater than A0A0H, it can preliminarily be identified as Chinese character encoding. Note: In data packets, you can only obtain the internal machine code. Zone-position code is an input code and does not exist inside computers.

II. What are the differences and relationships between GBK internal code, Unicode, and zone-position code? How to convert among them?

ANSI is an encoding format established by the American National Standards Institute. For example, the string "A汉" in ANSI encoding has the byte values 41 BA BA in memory: 'A' occupies one byte, while "汉" uses two bytes.

Here, BA BA is exactly the GBK internal code value. Let's first understand GBK encoding.

GB2312, GBK, and GB18030 are all Chinese-developed encoding standards (not used outside China), introduced in chronological order: GB2312 → GBK → GB18030. They are backward compatible. "GB" likely stands for "Guo Biao" (National Standard), and "K" may mean "Kuo Zhan" (Extended). These are formal specifications. To represent GB2312 in computer memory, each byte of the zone-position code must have 0x80 added to form the internal code. Adding 0x80 ensures the highest bit of each byte is 1, allowing memory to distinguish Chinese characters from ASCII (whose highest bit is always 0).

However, when extending GB2312 to GBK and GB18030, more encoding space was needed. Thus, GBK and GB18030 do not require the second byte’s highest bit to be 1. Instead, the first byte determines whether it's a single-byte ASCII or a double-byte GBK character. Note: GB2312, GBK, and GB18030 are backward compatible. For example, the character "汉" is encoded as BA BA in all three standards.

Moreover, GB2312 and GBK have not disappeared due to GB18030; they are still widely used in embedded devices because reducing font library size significantly cuts costs.

Back to ANSI: We now understand GBK (though it's unclear why GBK is preferred over GB18030—perhaps because "GBK" is shorter and easier to write). So what is ANSI? ANSI acts like a pointer with no inherent content. If it points to "Chinese encoding," it means GBK; if to "Indian encoding," it means something else. Thus, in China, ANSI means GBK; in Japan, it means XXX; in India, ???—but in Windows Notepad, they all appear as "ANSI." However, ANSI makes a small adjustment: as shown earlier, 'A' occupies only one byte in memory. So ANSI = ASCII + local encoding.

Unicode:

But how can we write Japanese characters in a Chinese document? This is where Unicode comes in. Whoever invented it, Unicode includes all written characters worldwide, solving the above issue. Programs written in Unicode by programmers can run on computers globally. In C, Unicode is represented using wchar_t.

UCS:

UCS is reportedly a parallel project to Unicode. Eventually, both projects reached consensus and are fully compatible. Thus, UCS can be treated as equivalent to Unicode.

UCS-2 (commonly referred to as UCS) uses two bytes per character, while UCS-4 uses four.

UTF-8:

(UCS Transformation Format) Why was UTF-8 created? One reason: In C and operating systems, 0x00 has special meaning (e.g., string termination). However, in raw Unicode encoding, a character might have a non-zero high byte and a zero low byte, causing C to mistakenly treat it as a string end. UTF-8 ensures that 0x00 does not appear in the encoded byte stream (except for the actual null character).

UTF-8 Encoding Rules:

UCS-2 (Hex)     UTF-8 Byte Stream (Binary)
0000 - 007F     0xxxxxxx
0080 - 07FF     110xxxxx 10xxxxxx  (Number of 1s after the first 1 indicates following bytes: 1 here)
0800 - FFFF     1110xxxx 10xxxxxx 10xxxxxx  (2 following bytes)

Due to this encoding scheme, UTF-8 does not require endianness detection, making it suitable for network transmission (reason not explained here).

BOM (Byte Order Mark): EF BB BF. We can use BOM to detect if a text file is UTF-8 encoded.

How to detect a text file's encoding when opening it?

Prompt the user to select the encoding type.
Guess the encoding based on certain rules.
Detect file header signatures:
- EF BB BF → UTF-8
- FE FF → UTF-16/UCS-2 (Unicode), little endian (e.g., a file with only 'A' contains FE FF 00 41)
- FF FE → UTF-16/UCS-2 (Unicode), big endian (e.g., FF FE 41 00)
- FF FE 00 00 → UTF-32/UCS-4, little endian
- 00 00 FE FF → UTF-32/UCS-4, big-endian

Zone-Position Code, GB (GBK) Code, Internal Code

Character	Zone-Position	GB Code	Internal Code
汉	1A1A	3A3A	BABA

GB Code = Zone-Position Code + 0x20 (per byte)
Internal Code = GB Code + 0x80 (per byte)

III. Summary of Chinese Character Encoding and Programming Issues

There are many Chinese character encodings. Common ones include UNICODE, GB (internal code), and GB2312-80 (zone-position code). UNICODE is the international character set standard, compatible only with ASCII.

The difference between GB (internal code) and GB2312-80 (zone-position code) is that GB (internal code) is represented as a 4-digit hexadecimal number, while GB2312-80 (zone-position code) uses a 4-digit decimal number. The conversion is as follows:

GB (internal code) = ((GB2312-80(zone-position)/100 + 160) << 8) | ((GB2312-80(zone-position) % 100) + 160)

1. How to convert or query Chinese character encoding?

For example, the character "汉": write it in Notepad and open the file with WinHex. The displayed code BABA is its GB (internal code). You can also calculate its zone-position code using the reverse method:

(High byte - 0xA0, Low byte - 0xA0) → Convert to decimal → GB2312-80 (zone-position code)

Note: 0xA0 = 160 (hex vs. decimal representation)

Example:

(0xBA - 0xA0) * 100 + (0xBA - 0xA0) = 26 * 100 + 26 = 2626 → This is the GB2312-80 zone-position code.

2. When programming with many Chinese characters, use WinHex's conversion feature to generate C code directly instead of typing manually.

For example, drag a document or image file into WinHex, right-click the data → Edit → Copy All → As C Source. The data in C format is copied to the clipboard and can be pasted into any document.

3. How to obtain the Unicode code of a Chinese character from text?

Use Notepad++, a free open-source editor (excellent for coding, with built-in syntax highlighting). Open a document in Notepad++, then:

Format → Convert to UCS-2 Big/Little Endian, then save. The file is now saved in Unicode encoding.

Open it in WinHex to view the Unicode code. Choosing big or little endian only affects byte order (big endian places high byte first, at lower memory address).

Below is an additional translated article and the standard Chinese zone-position code table to help understand various encoding formats:

////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////

"On Unicode Encoding" by fmddlmyy

Unicode is a character encoding method capable of accommodating all written languages. The encoding methods from ASCII, GB2312, GBK to GB18030 are backward compatible. Unicode, however, is only compatible with ASCII (more precisely, ISO-8859-1), not with GB codes. For example, the Unicode code for "汉" is 6C49, while its GB code is BABA.

This is a fun read for programmers—fun in the sense of easily understanding previously unclear concepts, gaining knowledge, like leveling up in an RPG game. The motivation for writing this article came from two questions:

Question 1:

Using Windows Notepad's "Save As," you can convert between GBK, Unicode, Unicode big endian, and UTF-8. For the same .txt file, how does Windows identify the encoding?

I long noticed that Unicode, Unicode big endian, and UTF-8 files have extra bytes at the beginning: FF FE (Unicode), FE FF (Unicode big endian), EF BB BF (UTF-8). But what standard defines these markers?

Question 2:

Recently, I found a ConvertUTF.c that converts among UTF-32, UTF-16, and UTF-8. I already knew about Unicode (UCS2), GBK, and UTF-8. But this program confused me—what's the relationship between UTF-16 and UCS2?

After researching, I finally clarified these issues and learned some Unicode details. I wrote this article for others with similar questions. I tried to make it easy to understand, assuming readers know what bytes and hexadecimal are.

0. Big Endian and Little Endian

Big and little endian are CPU methods for handling multi-byte numbers. For example, the Unicode code for "汉" is 6C49. When writing to a file, should 6C come first or 49? If 6C comes first, it's big endian; if 49 comes first, it's little endian.

The term "endian" comes from Gulliver's Travels, where a civil war erupted over whether to crack an egg from the big end (Big-Endian) or the small end (Little-Endian), causing six rebellions, one emperor's death, and another's dethronement.

We usually translate "endian" as "byte order," with "big endian" and "little endian" as "big-end" and "little-end."

1. Character Encoding, Internal Code, and Chinese Encoding

Characters must be encoded to be processed by computers. The default encoding used by a computer is its internal code. Early computers used 7-bit ASCII. To handle Chinese, programmers designed GB2312 for simplified Chinese and Big5 for traditional Chinese.

GB2312 (1980) includes 7,445 characters: 6,763 Chinese characters and 682 other symbols. The Chinese character area ranges from B0-F7 (high byte) and A1-FE (low byte), covering 72×94 = 6,768 code points, with 5 unused positions (D7FA-D7FE).

GB2312 supports too few characters. GBK1.0 (1995) expanded to 21,886 symbols, divided into Chinese and graphic symbol areas, with 21,003 Chinese characters. GB18030 (2000) replaced GBK1.0 as the official national standard, including 27,484 Chinese characters and major minority scripts like Tibetan, Mongolian, and Uyghur. Current PC platforms must support GB18030, but embedded products are exempt. Thus, phones and MP3s usually support only GB2312.

From ASCII, GB2312, GBK to GB18030, these encodings are backward compatible: the same character has the same code across standards, and later standards support more characters. English and Chinese can be uniformly processed. Chinese characters are identified by a high byte with the highest bit set to 1. Programmers refer to GB2312, GBK, and GB18030 as Double-Byte Character Sets (DBCS).

Some Chinese Windows systems still use GBK as the default internal code. GB18030 upgrade packs are available. However, the additional characters in GB18030 are rarely used, so "GBK" often refers to Chinese Windows internal code.

Some details:

GB2312 originally uses zone-position code. To convert to internal code, add A0 to both high and low bytes.
In DBCS, GB internal code is always stored in big-endian (high byte first).
Both bytes in GB2312 have their highest bit set to 1. But only 128×128 = 16,384 code points meet this. Thus, GBK and GB18030 low bytes may not have the highest bit set. However, this doesn't affect DBCS parsing: when reading a DBCS stream, encountering a byte with the highest bit set indicates the next two bytes form a double-byte character, regardless of the low byte's high bit.

2. Unicode, UCS, and UTF

As mentioned, ASCII → GB2312 → GBK → GB18030 are backward compatible. Unicode is only compatible with ASCII (ISO-8859-1), not GB codes. For example, "汉" is 6C49 in Unicode, BABA in GB.

Unicode is a character encoding method designed internationally to include all written languages. Its full name is "Universal Multiple-Octet Coded Character Set" (UCS), which can be seen as "Unicode Character Set."

According to Wikipedia (http://zh.wikipedia.org/wiki/), two independent groups developed Unicode: ISO (International Organization for Standardization) and Unicode Consortium (a software vendors' association). ISO developed ISO 10646; Unicode Consortium developed Unicode.

Around 1991, both realized the world didn't need two incompatible standards. They merged their work and collaborated on a single encoding table. Starting with Unicode 2.0, Unicode adopted the same character set and codes as ISO 10646-1.

Both projects still exist and publish standards independently. Unicode Consortium's latest version is Unicode 4.1.0 (2005). ISO's latest is 10646-3:2003.

UCS defines how to represent various scripts using multiple bytes. UTF (UCS Transformation Format) defines how to transmit these encodings. Common UTF formats include UTF-8, UTF-7, UTF-16.

IETF's RFC2781 and RFC3629 clearly and rigorously describe UTF-16 and UTF-8 encoding methods. IETF (Internet Engineering Task Force) maintains RFCs, the foundation of all Internet standards.

3. UCS-2, UCS-4, BMP

UCS has two formats: UCS-2 (2 bytes) and UCS-4 (4 bytes, using only 31 bits; highest bit must be 0). Some simple math:

UCS-2: 2^16 = 65,536 code points
UCS-4: 2^31 = 2,147,483,648 code points

UCS-4 divides the highest byte (with MSB=0) into 128 groups. Each group divides the second byte into 256 planes. Each plane divides the third byte into 256 rows, each with 256 cells (only the last byte differs).

Group 0, Plane 0 is called the Basic Multilingual Plane (BMP). Alternatively, UCS-4 code points with the top two bytes zero are BMP.

Removing the two leading zero bytes from UCS-4 BMP yields UCS-2. Adding two zero bytes to UCS-2 yields UCS-4 BMP. Currently, no UCS-4 characters are assigned outside BMP.

4. UTF Encoding

UTF-8 encodes UCS in 8-bit units. UCS-2 to UTF-8 conversion:

UCS-2 (Hex)     UTF-8 Byte Stream (Binary)
0000 - 007F     0xxxxxxx
0080 - 07FF     110xxxxx 10xxxxxx
0800 - FFFF     1110xxxx 10xxxxxx 10xxxxxx

Example: "汉" has Unicode code 6C49. Since 6C49 is in 0800–FFFF, it uses the 3-byte template: 1110xxxx 10xxxxxx 10xxxxxx.

6C49 in binary is 0110110001001001. Filling the template:

11100110 10110001 10001001 → E6 B1 89.

You can test this encoding in Notepad.

UTF-16 encodes UCS in 16-bit units. For UCS codes < 0x10000, UTF-16 equals the 16-bit unsigned integer. For ≥ 0x10000, a specific algorithm is used. However, since UCS-2 and UCS-4 BMP are always < 0x10000, UTF-16 and UCS-2 are currently nearly identical. But UCS-2 is just a scheme, while UTF-16 is for actual transmission, requiring byte order consideration.

5. UTF Byte Order and BOM

UTF-8 uses bytes as units, so no byte order issue. UTF-16 uses 16-bit units; before interpreting a UTF-16 text, you must know the byte order of each unit. For example, receiving byte stream 594E: is it "奎" (Unicode 594E) or "乙" (4E59)?

Unicode recommends using BOM (Byte Order Mark) to indicate byte order. BOM is a clever idea:

UCS defines a character "ZERO WIDTH NO-BREAK SPACE" with code FEFF. FFFE does not exist in UCS and should never appear in transmission. UCS suggests transmitting FEFF first.

If the receiver gets FEFF, the stream is big-endian; if FFFE, it's little-endian. Thus, "ZERO WIDTH NO-BREAK SPACE" is called BOM.

UTF-8 doesn't need BOM for byte order, but can use it to indicate encoding. The UTF-8 encoding of FEFF is EF BB BF. So, if a stream starts with EF BB BF, it's UTF-8.

Windows uses BOM to mark text file encodings.

6. Further References

Main reference: "Short overview of ISO-IEC 10646 and Unicode" (http://www.nada.kth.se/i18n/ucs/unicode-iso10646-oview.html).

Two other good resources (not read, as my questions were answered):

"Understanding Unicode A general introduction to the Unicode Standard" (http://scripts.sil.org/cms/scripts/page.php?site_id=nrsi&item_id=IWS-Chapter04a)
"Character set encoding basics Understanding character set encodings and legacy encodings" (http://scripts.sil.org/cms/scripts/page.php?site_id=nrsi&item_id=IWS-Chapter03)

I've written software packages for converting between UTF-8, UCS-2, and GBK, including versions with and without Windows API. I may publish them on my homepage (http://fmddlmyy.home4u.china.com) later.

I wrote this article only after fully understanding everything, expecting it to take minutes. But wording and detail-checking took hours—from 1:30 PM to 9:00 PM. I hope readers benefit.

//////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////

GB = (Hex)(GB2312(H) + 160) + (Hex)(GB2312(L) + 160)

People's Republic of China National Standard

Chinese Character Coded Character Set for Information Interchange

Basic Set

GB 2312-80

(Zone-position code table follows, unchanged)

Design and Implementation of AM5728 DSP+ARM-Based Logistics Robots for Automated Sorting and Delivery, AGV

Leo Liu — Sun, 24 May 2026 07:03:46 +0000

Originally published on the Sienovo Engineering Blog. Sienovo is the overseas brand of 深圳信迈 (Shenzhen Xinmai), building edge AI computing solutions for industrial video analytics.

AM5728 DSP+ARM-based logistics robot enabling automated sorting and delivery

Physical view of Amazon logistics robots:

Function Overview and Application Areas

China's logistics industry started relatively late and lags behind in development level, especially in the application of information technology, where it significantly trails developed countries. Many advanced technologies and equipment widely adopted abroad are rarely used domestically. This gap results in hundreds of billions of yuan in economic losses annually due to inefficiencies in logistics. Achieving logistics informatization has thus become an inevitable trend for the modern development of China's logistics sector.

The implementation of barcode identification systems in express logistics has enhanced the informatization level of logistics enterprises, significantly improving operational efficiency, reducing costs, and minimizing errors.

At the front end of the logistics workflow—immediately after unloading—industrial machine vision technology is employed to automatically scan parcel label barcodes.

This solution uses smart cameras to capture images of shipping labels. The captured images are transferred to an image processing board for algorithmic analysis to extract barcode data. This barcode data is then combined with existing weight and volume information from volumetric weighing machines. Finally, the integrated data is transmitted to a central server via 2G/3G/4G networks, enabling automated sorting at distribution centers.

Applications include automated sorting and delivery systems for major Chinese courier companies such as Yunda, SF Express, YTO Express, and STO Express.

Overall Design and Key Modules

▼ Image Acquisition and Processing Hardware System Based on Core Module

As shown above, the XM5728-DIK-V3 is compact and features a no-edge-design, with all functional interfaces brought out via pin headers. However, the baseboard's built-in functions alone do not fully meet customer requirements. Therefore, customers designed a custom carrier board, extending required functionalities through MIOe expansion interfaces and direct pin connections from the main module. By using PCB traces instead of cables, the design efficiently integrates all desired functions into the carrier board, quickly and effectively meeting interface, thermal dissipation, mechanical structure, and shock resistance requirements. Sienovo provides comprehensive technical support, including schematic review and debugging assistance.

Visual Inspection Principle of Sorting System

Visual inspection consists of two parts: detecting the color and shape of packages, and recognizing barcodes on shipping labels.

1. Package Color and Shape Detection

A color camera is used to identify packages. When a package reaches directly beneath the camera, the camera is triggered to capture an image. The AM5728 processor acquires and analyzes the image using algorithms.

The recognition algorithm includes the following steps:

❶ Noise reduction on the captured image;

❷ Image segmentation based on color features to separate the package region from the background. The shape characteristics of the package are calculated from the edge features of the segmented region;

❸ Within the package area in the image, connected component analysis combined with the label's color and rectangular shape features is used to locate the shipping label. After excluding the label region, the remaining package area is analyzed to determine the package's color characteristics.

2. Barcode Recognition

Four cameras simultaneously capture images of the package across its horizontal surface. The number of camera captures is determined based on the previously obtained shape characteristics, ensuring that every longitudinal section of the package surface is imaged.

The barcode recognition algorithm includes:

❶ Noise reduction on the captured images;

❷ Detection of barcode presence in the processed image;

❸ Precise localization and decoding of barcode regions, with results sent to the host computer interface.

Core Technologies of Fixed-Mount Imaging Smart Cameras – Key Features:

Core Module Based on TI AM5728 Multi-Core Heterogeneous Processor

1. Dual ARM Cortex-A15 + Dual C66x VLIW Floating-Point DSP C66X Heterogeneous Hybrid Processor

➢ Features 2x Cortex-A15 @ 1.5GHz, 2x C66x DSP @ 750MHz, 2x Cortex-M4, and 2x PRU

➢ Uses industrial-grade precision BTB connectors to ensure signal integrity, enabling stable, reliable, and easy installation/removal

➢ Supports up to 6 video inputs and 3 video outputs

➢ Supports up to 8 McASP interfaces

➢ Supports up to 2 CAN interfaces

➢ Supports up to 10 UARTs

➢ Supports up to 3 LCD interfaces

➢ Supports up to 4 SPI interfaces

➢ Supports up to 3 MMC interfaces

➢ Supports up to 5 I2C interfaces

➢ Supports up to 2 Gigabit Ethernet ports

➢ Supports 1 SATA-2 interface

➢ Supports 1 USB 3.0 interface

➢ Supports 2 USB 2.0 interfaces

➢ Supports 1 HDMI interface

➢ Supports 1 PCIe 3.0 interface, configurable in 1×2-lane or 2×1-lane mode

➢ Supports Linux 4.14.67, DSP RTOS, and real-time RT-Linux systems

➢ Operating temperature range: -40°C to +85°C (industrial grade)

2. Multi-Core Heterogeneous Development Platform

Provides command-line and Qt-based development examples, along with OpenCV, OpenCL, and OpenGL development guides. Also includes environment setup and development tutorials for DSP and PRU.

3. High-Performance DSP

The Sienovo AM5728 development board integrates a real-time C66x digital signal processor (DSP) running at 750MHz, widely used in audio processing, video processing, and complex data algorithm applications. Sienovo provides an OpenCV vision library that includes reference demos combining OpenCV with DSP for edge detection, face recognition, and integration with OpenCL and OpenGL, enabling rapid customer development.

4. PRU Programmable Real-Time Unit

Equipped with two dual-core PRU-ICSS modules supporting industrial protocols such as EtherCAT, EtherNet/IP, POWERLINK, PROFIBUS, PROFINET RT/IRT, and SERCOS III. These enable real-time fieldbus communication and provide developers with ultra-low latency and real-time control capabilities essential for industrial applications, with timing precision down to the nanosecond level.

5. Rich Multimedia Capabilities

Integrated SGX5443-D and GC320 2D graphics accelerators support advanced graphical user interfaces. The Cortex-M4 IPU2 graphics processing unit is dedicated to IVA hardware video encoding/decoding, supporting hardware decoding of multiple formats including H.264 and MPEG-4. It enables simultaneous multi-channel video decoding at up to 1080p 60fps. Supports MJPEG hardware encoding/decoding. The VPE supports hardware-based image and video scaling and color space conversion.

6. Extensive Communication Interfaces

■ Dual Gigabit Ethernet interfaces supporting Ethernet/IP and PROFINET protocols;

■ RS232/RS485 bus support for connection to PLCs, computers, and other systems;

■ Onboard WiFi & Bluetooth combo module and 4G communication support.

Achieved Technical Performance Metrics

In logistics applications, barcodes on shipping documents must be scanned at multiple stages—from pickup to sorting centers, and from sorting centers to final delivery.

The barcode reader described in this article eliminates manual operation by being mounted as a fixed station on automated conveyor lines, enabling automatic barcode localization and decoding. Based on fixed-mount imaging smart cameras, this system significantly improves both efficiency and accuracy throughout the entire logistics process.

Reference solution: Sienovo's XM5728-IDK-V3 development board.

System Architecture and Mechanical Structure

From the outside, each side of the Kiva robot's housing features an infrared sensor array and a pneumatic bumper for collision detection and cushioning. The housing also includes a charging port and a series of status indicator lights.

Each Kiva robot has three independent degrees of freedom: two drive wheels and one rotating lift motor. When the lift motor operates, the two drive wheels rotate in opposite directions, resulting in the platform rising along the ball screw without rotating relative to the ground. Compared to traditional high-load linear actuation methods such as hydraulic or scissor lifts, Kiva's wheel-based lifting mechanism is clearly simpler and more reliable.

The top of the lift platform consists of robust X-shaped aluminum castings made from 319-grade general-purpose aluminum. Each casting undergoes secondary precision machining to create reference surfaces and threaded holes. This manufacturing process is commonly used in equipment such as automotive engines and hydraulic pumps.

Left: Internal structure of Kiva with housing removed, showing the infrared array, wireless module, lifting mechanism, and lift motor. Right: Top view of Kiva, showing the lifting mechanism and battery.

Each infrared sensor has its own dedicated filtering chip and communicates via a serial bus. In the image, you can see the motor and large gear used in the lifting module. Near the bottom of the robot, four lead-acid batteries are installed.

Housing

The orange, streamlined plastic housing is vacuum-formed from ABS material and includes numerous secondary machined features. The vacuum-forming machine and CNC milling tools used to manufacture Kiva were likely large and expensive. This version of the Kiva housing is both complex and costly; future versions are expected to adopt fully injection-molded structures.

Collision Sensors

For a large, streamlined housing like Kiva's, implementing a traditional integrated collision sensor is extremely difficult. Kiva engineers devised a clever, low-cost solution: inflating ethylene/rubber tubes and using a simple pressure sensor. Any change in internal pressure immediately triggers the robot to stop all motion. The black box on the right side of the image processes pressure signals and all infrared sensor data, simplifying communication protocols and wiring with the main controller.

Lifting Mechanism

Gearbox, lift motor, and large-diameter ball screw. The lifting mechanism uses a custom ball screw connected to the motor via a standard nylon gear. The same Pittman motor used for lifting is also used for the two drive wheels, capable of delivering approximately 3 N·m of torque and 1 kW of stall power. The motor output shaft connects to a 25:1 Japanese Brother gearbox, producing 46 N·m of torque at 72 rpm. This gearbox costs around $1,000 per unit, though bulk purchasing would likely reduce the price significantly.

Chassis and Drive Wheels

After removing the lifting mechanism, flipping the robot reveals the drive components on the underside. The two drive motors and gearboxes are identical to the lift motor setup. Two custom wheels form a differential drive system capable of in-place rotation.

Three sand-cast aluminum components make up most of the robot's chassis. They are connected with simple U-shaped pin clamps, forming a passive dual suspension system. These aluminum parts are also made from 319 alloy and use a casting followed by precision machining process. Note the machined heat dissipation structures on the chassis, on the back of which are mounted the large MOSFETs of the motor drivers. This design naturally uses the chassis for maximum heat dissipation efficiency.

U-shaped pin clamps connecting the suspension, and heat sinks on the chassis.

Electronics

Designing reliable electronics capable of powering Kiva's three high-power motors and numerous sensors over long periods is a critical challenge.

Battery Module

Battery wiring harness and charging port (top and side views). The entire system is powered by four series-connected 12V, 28Ah lead-acid batteries. Two of the four batteries are equipped with thermocouples to prevent overheating. When battery levels are low, the robot automatically disengages from central controller commands and returns to the charging station. The charging station is designed with ample clearance to ensure reliable charging.

Cameras and Imaging Module

Upward- and downward-facing cameras, both mounted inside the ball screw assembly. Located within the lifting mechanism is one of the key components of the Kiva system: a custom dual-camera imaging module. One camera looks downward to identify 2D barcodes on the warehouse floor, while the other looks upward at the underside of shelves. Each camera is equipped with six red LEDs for illumination. Sandwiched between the two cameras is the image processing module, centered around an ADI ADSP-BF548 Blackfin multimedia processor that acquires data via high-speed serial interface and performs data matrix detection.

Mainboard and Subboards

The main logic module shown above orchestrates all operations. The motor driver board is powered by 48V DC from the battery; logic components are powered by a separate filtered power rail. The three-phase brushless DC (BLDC) motor drivers are fully custom, driven by a Lattice LFXP6C FPGA (hidden beneath the mainboard).

All three motor drivers include current sensors (suggesting FOC control), encoders, and six full-bridge MOSFETs (cooled via the chassis). A subboard hosting the FPGA handles coordination of the wireless module, imaging unit, emergency braking, infrared/pressure sensors, power management, and motor drivers, significantly reducing the load on the mainboard. The MCU is a 32-bit, 400MHz Freescale MPC5123, likely running PowerPC Linux. Two Ethernet ports connect to the wireless module and firmware storage via a Microchip KSZ8993 switch. The only off-the-shelf electronic component in the entire robot is the communication module: a Soekris Engineering Net4526 dual-antenna router running a single Winstron NeWeb CM9 wireless module, connected to the mainboard via Ethernet.

The remarkable lifting module

This generation of Kiva robots features many clever design elements, but the standout is undoubtedly the lifting module. It must raise and lower loads up to 1,000 pounds (about half a ton) while maintaining perfect parallelism with the ground—ideally suited for a ball screw mechanism. Typically, commercial ball screws are solid and rarely exceed five centimeters in diameter. Kiva's custom version, however, has an outer diameter of 28 centimeters and is hollow with internal threading. The two housing bearings are aluminum, cast and then precision-machined like the chassis components. Both parts are surface-oxidized, providing excellent lubrication and corrosion resistance. The inner housing is fixed, acting as a ball nut, with an overmolded ring structure to constrain the balls. The outer housing rotates, with its inner surface contacting the balls and its outer surface connected to the lift motor via gears.

Considering manufacturing complexity, the entire lifting mechanism is estimated to cost around $1,000.

Kiva Systems is one of the few companies capable of seamlessly integrating sophisticated hardware and software into a unified solution, having built a system that profoundly transforms how we buy, sell, and live. Clearly, Kiva employs a team of exceptionally skilled hardware engineers—likely a major reason why Amazon acquired the company in 2012 for a staggering $775 million. This article only discusses the robot itself, but it's important to note that this is just a small (albeit impressive) part of the entire Kiva solution.

Domestic Virtual Instruments: Zynq-based Radar 10Gbps High-Speed PCIe Data Acquisition Card Solution (Part 2) Hardware Design

Leo Liu — Sat, 23 May 2026 06:52:33 +0000

Originally published on the Sienovo Engineering Blog. Sienovo is the overseas brand of 深圳信迈 (Shenzhen Xinmai), building edge AI computing solutions for industrial video analytics.

This article details the hardware design of a Zynq-based 10 Gbps PCIe radar data acquisition card, covering chip selection, power architecture, clock distribution, high-speed interfaces, DDR3 memory, and the design of both the ADC daughter card and fiber-optic daughter card. Engineers working on FPGA-based data acquisition systems will find a practical reference for how to partition carrier-board and daughter-card responsibilities, select appropriate power management ICs, and configure the Xilinx Zynq-7100 SoC for high-throughput signal capture.

3.1 Introduction

The hardware design of the acquisition card is the foundation for realizing its capture functions. A well-conceived hardware design makes the acquisition feature easier to implement and streamlines subsequent software development. This chapter builds on the hardware architecture described in Chapter 2 and provides a detailed walkthrough of chip selection, configuration, and specific circuit design for both the carrier board and the daughter cards. The overall block diagram for the carrier board and daughter cards is shown in Figure 3.1.

3.2 Carrier Board Hardware Design

3.2.1 Main Control Chip Selection and Configuration Circuit

(1) Chip Selection and Analysis

Based on the hardware design analysis from Chapter 2, the main control chip must satisfy the following requirements:

At least 12 pairs of high-speed GTX transceivers — 4 pairs for fiber optic links and 8 pairs for PCIe.
Sufficient on-chip BRAM resources to implement FIFO buffering at the data interfaces.
Native support for the Aurora serial protocol and PCIe 2.0.
General-purpose peripheral interfaces (Ethernet, SPI, UART, CAN) for ease of debugging and porting.

Considering all of these constraints, the design selects a chip from the Xilinx Zynq-7000 SoC family. The Zynq-7000 series integrates a dual-core ARM Cortex-A9 processing system (PS) with a Xilinx Kintex-7 FPGA programmable logic (PL) fabric on the same die, giving both hard-processor control and high-speed programmable I/O in one package. After comparing the resource tables for the product family (Table 3.1), any device at the Zynq-7135 or below can meet the design's requirements. The final selection is XC7Z100FFG900-2 from the Zynq-7100 sub-family.

The XC7Z100FFG900-2 IOBANK layout (Figure 3.2) is organized as follows:

Bank 9–13, 33–35 — PL-side general I/O pins.
Bank 109–112 — PL-side high-speed GTX transceivers; each bank provides four pairs of high-speed differential transceivers.
Bank 500–502 — PS-side pins, providing standard interfaces such as Gigabit Ethernet, SPI, CAN, and UART.

White squares in the diagram represent power, ground, or configuration pins.

(2) Power Supply Configuration Circuit

The Zynq-7100FFG900 PL and PS sections each require different I/O supply voltages depending on bank type and performance class. PL banks are divided into HR (High Range) and HP (High Performance) categories:

HR banks (9–13): support I/O voltages from 1.2 V to 3.3 V.
HP banks (33–35): support I/O voltages from 1.2 V to 1.8 V only.

Beyond I/O voltages, the PL fabric also needs a core supply, an auxiliary supply, a GTX transceiver core supply, and a GTX termination supply. The PS side requires its own internal voltage, auxiliary voltage, and MIO voltage rails. The full supply voltage breakdown is summarized in Table 3.2.

(3) Clock Configuration Circuit

The clock distribution circuit is another critical configuration element in the Zynq-7000 design. All timing and control signals are derived from external clock references; excessive clock jitter or drift degrades overall system performance and can cause the system to stop functioning entirely. Active crystal oscillators are used as clock sources throughout.

The Zynq-7000 family has three distinct clock domains:

PS clock: provided by a 33.333333 MHz active oscillator with a single-ended output.
PL clock: provided by a 100 MHz active oscillator with a differential output.
High-speed GTX clock: frequency is protocol- and data-rate-dependent and is generated by a programmable clock chip.

The PS and PL clock circuits are shown in Figure 3.3(a) and 3.3(b) respectively.

Because high-speed protocols operate at a variety of line rates, the GTX reference clock frequency must be configurable. This design uses the TI CDCM61002 programmable clock IC, which accepts a single-ended input in the range of 21.875–28.57 MHz, locks an internal high-performance VCO running at 1.75–2.05 GHz, and outputs the VCO divided by 1, 2, 3, 4, 6, or 8. It provides two pairs of differential output clocks and supports both LVCMOS and LVDS output levels.

In this design a 25 MHz reference is fed to the CDCM61002, and DIP switches are used to set the divider ratio by driving the chip's configuration pins to different logic levels. This yields output frequencies spanning 62.5 MHz to 625 MHz, which covers the reference clock requirements of most high-speed serial protocols encountered in radar data acquisition.

The high-speed clock configuration block diagram is shown in Figure 3.4, and the DIP-switch configuration versus output-frequency mapping is listed in Table 3.3.

(4) High-Speed Interface Circuit

The XC7Z100FFG900 provides 16 GTX transceiver pairs in total. This design uses 12 of them: 4 pairs are routed to the FMC connector (bank 110) for the daughter-card optical link, and 8 pairs are routed to the PCIe edge connector (banks 111 and 112).

The high-speed interface block diagram is shown in Figure 3.5.

(5) Peripheral Configuration and Boot Circuit

Like all Xilinx FPGAs and SoCs, the Zynq-7000 is volatile: the PL bitstream and PS software are lost when power is removed. The design therefore provides multiple non-volatile storage options:

Two QSPI Flash chips — Cypress S25FL256S, 256 Mb each, connected to PS bank 500. Maximum read/write clock is 133 MHz.
One SD card interface — connected to PS bank 501, used for storing the boot image or OS.
One eMMC device — Micron MTFC8GAKAJCN-4M, 8 GB capacity, 50 MHz clock, connected to PL bank 35 and accessed via the EMIO feature.

The QSPI Flash connection block diagram is shown in Figure 3.6, and the eMMC / SD card connection diagram is shown in Figure 3.7.

For remote control and firmware updates over the network, a standard RJ-45 Gigabit Ethernet port is provided. The PHY is the Marvell 88E1518, a tri-speed (10/100/1000 Mbps) Ethernet transceiver connected to PS bank 501. The network port and PHY block diagram is shown in Figure 3.8.

3.2.2 DDR3 SDRAM Selection and Configuration

Both the PS and PL domains require external DDR3 SDRAM. The PS-side DRAM provides a working memory region during boot and OS operation, while the PL-side DRAM serves as the high-bandwidth data cache for buffering incoming high-speed acquisition data before it is transferred to the host over PCIe.

The design uses Micron MT41K256M16TW-107 DDR3 SDRAM. Two chips are populated on each of the PS and PL sides, giving a 32-bit data bus width per domain (2 × 16-bit devices) and a total capacity of 1 GB per domain. Key performance figures:

PS-side DDR3: up to 1066 MHz operating frequency.
PL-side DDR3: up to 1600 MHz operating frequency.

The PL-side DDR3 is connected to HP banks 33 and 34, which offer cleaner power rails and tighter I/O timing margins compared to HR banks — important for extracting maximum DDR3 performance. The PS-side DDR3 connects to the dedicated memory interface bank 502. The DDR3 block diagrams for both domains are shown in Figure 3.9.

3.2.3 High-Speed Interface Circuit

The carrier board provides two classes of high-speed interfaces: FMC and PCIe.

As a general PCB layout rule, 0.1 µF AC-coupling capacitors are placed at every high-speed differential signal pin to block DC offsets. Differential pairs within a single channel are routed as tightly coupled, parallel traces on the same layer to minimize impedance discontinuities, and vias between layers are kept to a minimum. Across multiple channels, all differential pairs are length-matched to ensure phase-aligned, synchronous data reception.

FMC (FPGA Mezzanine Card) is the VITA 57.1 standard interconnect between a carrier board and a daughter card. The daughter card typically carries the male connector and the carrier board the female connector. FMC supports transfer rates up to 10 Gbps per lane, with a theoretical aggregate throughput approaching 40 Gbps. Two variants exist:

LPC (Low Pin Count): 160-pin connector.
HPC (High Pin Count): 400-pin connector, used here because it provides the required number of GTX transceiver lanes.

In addition to the high-speed data lanes, the FMC connector also carries a JTAG chain, an I²C management bus, and dedicated power pins that allow the carrier board to supply the daughter card directly. This decouples the daughter-card logic design from the carrier board's general-purpose FPGA I/O pin constraints and makes the daughter-card design more portable.

The PCIe interface (Figure 3.10) uses a gold-finger edge connector that plugs directly into a host PC motherboard slot. RXD pins receive data from the host; TXD pins transmit to the host. AC-coupling capacitors (0.1 µF) are required on the TXD differential lines. The PCIe lane width is selectable — X1, X2, or X8 — by connecting the appropriate PRSNT_xx pin to PCIE_PG. This design implements PCIe X8 mode, requiring PRSNT_x8 to be tied to PRSNT.

3.2.4 Power Circuit

A well-designed power delivery network is critical: excessive output ripple causes instability, performance degradation, and can permanently damage components. The PC motherboard supplies the PCIe slot at 12 V, so the acquisition card's primary input is 12 V DC, which is then down-converted by on-board power management ICs to supply each subsystem at its required voltage.

Power management ICs must not be operated at or beyond their rated output current, since that leads to voltage droop and thermal runaway. The Zynq-7000 is a low-power SoC; its combined PS + PL current draw in this application is within a few amperes, which is well within the budget of the chosen regulators.

The design uses a core board + baseboard split. The main control chip sits on the core board. Performing all power conversion on the baseboard would complicate the baseboard layout and tightly couple the core board to a single baseboard design. Instead, the 12 V rail is brought onto the core board and converted there — making the core board self-contained and reusable with other baseboards.

The baseboard still needs modest power conversion for its own peripherals (USB, Ethernet PHY, FMC power delivery). That conversion is handled by a TI DC/DC converter with an input range of up to 18 V and a programmable output range up to 7 V, rated for up to 3 A output — sufficient for the baseboard's peripheral load. The baseboard power block diagram is shown in Figure 3.11.

The core board requires multiple supply voltages — 0.75 V, 1.0 V, 1.5 V, 1.8 V, 2.0 V, 3.3 V, and 5.0 V — for the Zynq SoC, DDR3 SDRAM termination and core, and other peripherals. The full core-board power conversion tree is shown in Figure 3.12.

Key power IC choices on the core board:

12 V → 1 V (Zynq PL core): TI TPS53355. Conversion input range 1.5–15 V; bias supply input range 4.5–25 V; adjustable output down to a fraction of a volt; maximum output current 30 A.
12 V → intermediate rail: TI DC/DC converter with 6 A output capability and output adjustable from 0.76 V upward.
5 V → lower voltages (1.5 V, 1.8 V, etc.): MPS MP2143, input range 2.5–5.5 V, output range 0.6 V to Vin − 0.5 V, 3 A output capability.

All selected ICs were verified to meet the actual load current demands with margin, ensuring stable operation across the full operating range.

3.3 Daughter Card Circuit Design

3.3.1 AD Daughter Card Circuit Design

The ADC daughter card performs analog-to-digital conversion of the incoming radar IF/baseband signal and delivers the digitized samples to the Zynq chip over the FMC interface.

The ADC is the TI ADC12D800, a dual-channel, 12-bit converter with a maximum sample rate of 1.6 GSPS (interleaved). Its output can be configured either as 12 differential output pairs or as parallel data lines. The two independent channels are designated I and Q, each with its own dedicated data bus, supporting both single-edge (SDR) and double-edge (DDR) clocking modes. The ADC's clock and control interface signals are summarized in Table 3.5.

The ADC12D800 also provides an SPI interface for extended control mode, enabling software configuration of offset trim, calibration, and other parameters. This design does not use the SPI configuration path, so that interface is not discussed further here.

The analog input is AC-coupled — the incoming RF/IF signal enters through an SMB coaxial connector, passes through the matching and balun network, and is converted to a differential signal pair before entering the ADC's differential input pins. The input circuit is shown in Figure 3.13.

The ADC12D800's maximum operating voltage is 2.2 V. This design powers it at 1.9 V, derived from the 3.3 V supply available on the FMC connector using a TI LP38503-ADJ LDO regulator (input range 2.7 V to 6 V, output range adjustable up to 5.0 V, maximum output current sufficient for the ADC load).

All ADC control lines and data lines are brought out to the FMC connector, with equal-length routing enforced on the data bus to satisfy timing closure for synchronous parallel capture. The complete AD daughter card block diagram is shown in Figure 3.14.

3.3.2 Fiber-Optic Daughter Card Circuit Design

The fiber-optic daughter card provides the physical-layer bridge between the optical fiber link and the FMC connector on the carrier board. Its primary component is a QSFP (Quad Small Form-factor Pluggable) connector (part number 1761987-9), which exposes 4 high-speed differential transceiver lanes in a compact, hot-pluggable package well-suited for multi-lane fiber-optic links.

The optical transceiver module used is an SR4-40G QSFP+ module, offering an aggregate line rate of 40 Gbps (4 × 10 Gbps) over multi-mode fiber with a reach of up to 150 m. The QSFP cage requires a 3.3 V supply, which is drawn directly from the FMC power pins provided by the carrier board. The four high-speed differential pairs from the QSFP connector are routed directly to the FMC HPC connector, where they connect to the GTX transceiver lanes in bank 110 of the Zynq-7100.

This passthrough architecture keeps the fiber daughter card simple: it performs no signal conditioning beyond the optical-to-electrical conversion inside the QSFP module, and the Aurora protocol framing and 8b/10b encoding are handled entirely within the Zynq PL fabric. Equal-length routing and controlled-impedance traces are used on the high-speed differential lines between the QSFP cage and the FMC connector to preserve signal integrity at 10 Gbps per lane.

Summary

The hardware design presented in this chapter covers every major subsystem of the Zynq-based 10 Gbps PCIe acquisition card:

Subsystem	Key Choice
Main control SoC	XC7Z100FFG900-2 (Zynq-7100, dual Cortex-A9 + Kintex-7 PL)
GTX transceivers used	12 of 16 pairs (4 FMC / 8 PCIe)
PS DDR3	2 × Micron MT41K256M16TW-107, 32-bit / 1 GB, 1066 MHz
PL DDR3	2 × Micron MT41K256M16TW-107, 32-bit / 1 GB, 1600 MHz
GTX reference clock	TI CDCM61002, 62.5–625 MHz DIP-selectable
Non-volatile storage	2 × Cypress S25FL256S QSPI Flash + SD card + 8 GB eMMC
Ethernet PHY	Marvell 88E1518 (10/100/1000 Mbps)
ADC daughter card	TI ADC12D800, 12-bit, 1.6 GSPS, dual I/Q channels
Fiber daughter card	QSFP SR4-40G, 40 Gbps aggregate over 150 m MMF
Primary power input	12 V DC (from PCIe slot), down-converted on core board

The carrier-board / daughter-card split via the FMC HPC connector gives this platform its modularity: the same Zynq carrier can be paired with different signal-conditioning daughter cards (ADC, fiber, DA, etc.) simply by swapping the mezzanine module, while the Zynq PL firmware adapts to the new interface. This approach is characteristic of the VITA 57.1 ecosystem and substantially reduces the NRE cost of iterating on front-end hardware.

STM32 ARM+FPGA Servo Control System (Part Two) Software and FPGA Design

Leo Liu — Sat, 23 May 2026 06:52:31 +0000

Originally published on the Sienovo Engineering Blog. Sienovo is the overseas brand of 深圳信迈 (Shenzhen Xinmai), building edge AI computing solutions for industrial video analytics.

A complete servo system involves numerous modules, making it impractical to detail each one. This article focuses on the more challenging FPGA implementation, providing a brief overview of the ARM-side workflow. The FPGA is central to the control system, handling critical functions like the Field-Oriented Control (FOC) algorithm, current sampling, and encoder sampling. The performance of these FPGA-based algorithms directly dictates the quality of motor control. Due to the inherent parallelism and real-time constraints of FPGA operation, careful consideration of design timing and the coordinated scheduling of various tasks is paramount, significantly increasing design complexity. Below, we will delve into the design specifics of key FPGA program parts, primarily covering the Space Vector Pulse Width Modulation (SVPWM) algorithm module, the BISS-C encoder feedback module, and the Tamagawa encoder feedback module.

4.1 SVPWM Algorithm Program Design

As established in prior discussions, the SVPWM algorithm's workflow is a sequential process designed to generate the precise voltage required for motor control. This process begins by decoding the stationary frame voltage components, U_alpha and U_beta, to accurately determine the current sector of the voltage vector. Following sector identification, the system calculates the active time required for the basic voltage vectors within that specific sector. The next step involves determining the precise time points for PWM comparison. Finally, pulse width modulation is performed, incorporating a crucial dead-time to generate complementary PWM waves. These waves then control the inverter switches, ultimately producing the desired sinusoidal voltage waveform for the motor.

4.1.1 Software Module Design

The SVPWM module's functional structure is designed to manage the generation of these control signals. The module interacts with several key signals:

sys_clk: The system clock, operating at a frequency of 50MHz, provides the timing backbone for all operations.
sys_rst: The system reset signal, which is active low, initializes the module to a known state.
svpwm_en: The module enable signal, which activates or deactivates the SVPWM generation process.
ualpha, ubeta: These are the output signals from the inverse Park transformation module, representing the voltage components in the stationary reference frame.
Other: This refers to the six complementary PWM waves (three pairs) that drive the inverter's power switches.

The underlying principles and computational formulas for this module are detailed in Chapter 2 and will not be reiterated here. However, a critical design consideration is the generation of complementary PWM waves with a precisely defined dead-time. This dead-time is essential to account for the mechanical switching delays inherent in the inverter's power devices (e.g., MOSFETs or IGBTs). Without adequate dead-time, there is a risk of simultaneously turning on both the upper and lower switches of an inverter leg, leading to a direct short circuit across the DC bus, which can damage the inverter and pose a safety hazard.

The principle for generating complementary PWM waves with dead-time involves using the calculated PWM wave as a reference. A suitable dead-time is then configured based on the specific characteristics of the inverter hardware. Assuming the inverter input signal is active high:

When the reference PWM signal transitions from low to high (indicating the upper switch should turn on), the lower switch is first commanded to turn off. After a delay equal to the defined dead-time, the upper switch is then commanded to turn on.
Conversely, when the reference PWM signal transitions from high to low (indicating the upper switch should turn off), the upper switch is first commanded to turn off. After a delay equal to the defined dead-time, the lower switch is then commanded to turn on.

This mechanism ensures that there is a brief period where both switches in a leg are off, preventing shoot-through. For example, considering the U-phase, if u_pwm is the calculated reference PWM wave, u_pwm1 would be the PWM wave for the upper switch, and u_pwm2 for the lower switch. The dead-time would be visible as a short delay between u_pwm1 turning off and u_pwm2 turning on, and vice-versa.

4.1.2 Simulation Test

To validate the functionality of the SVPWM module, a simulation was conducted. The simulation results demonstrated that by assigning arbitrary values to u_alpha and u_beta, and then generating a single clock cycle svpwm_en signal, the system correctly produced u_pwm as the reference PWM wave. Crucially, based on this u_pwm signal, the module successfully generated the complementary PWM waves, u_pwm1 and u_pwm2, with the expected dead-time clearly visible between them. This simulation confirmed that the SVPWM module is capable of generating the required complementary PWM waves with dead-time, ensuring safe and effective inverter operation.

4.2 BISS-C Protocol Program Design

The BISS (Bi-directional Synchronous Serial) protocol, developed by iC-Haus, is a robust bi-directional serial interface specifically designed for sensors. It incorporates CRC (Cyclic Redundancy Check) functionality to ensure the integrity and correctness of transmitted data. Compared to other similar communication protocols, BISS offers significant advantages in terms of compatibility and network structure. It achieves high speeds and low latency, comparable to SSI (Synchronous Serial Interface), with communication rates reaching up to 10Mbps. Furthermore, the BISS protocol allows for flexible adjustment of data length according to specific application requirements, providing excellent scalability.

BISS protocol supports two operating modes. The design discussed here utilizes the sensor mode, where the FPGA acts as the master, sending position request commands to the encoder, and the encoder (slave) responds with relevant information. In sensor mode, the BISS-C data format is structured as a series of bits transmitted over the SLO (Slave Output) data line, synchronized by the MA (Master) clock signal generated by the host.

4.2.1 BISS-C Protocol Data Structure

A BISS-C data frame is composed of eight distinct parts:

Ack (Acknowledgement) Phase: The encoder signals its readiness.
Start Bit: A single bit indicating the start of data transmission.
One '0' Bit: A fixed zero bit following the start bit.
Position Data: A variable number of bits representing the encoder's absolute position.
Error Bit: A single bit indicating an error condition in the encoder.
Warning Bit: A single bit indicating a warning condition in the encoder.
CRC Check Code: A six-bit CRC checksum for data integrity verification.
Timeout Phase: The concluding phase of the communication cycle.

The communication request cycle proceeds as follows:

Idle State: When no position request is active, the MA clock signal is held high. The host monitors the encoder's SLO data line. If SLO is high, it indicates the encoder is ready for communication.
Start MA Clock: Once the encoder is ready, the host begins transmitting a fixed-frequency MA clock signal to the encoder.
Encoder Acknowledgment: Upon detecting the second rising edge of the MA clock signal from the host, the encoder responds by pulling its SLO line low.
Host Switches to Receive: The encoder pulling the SLO line low signals to the host that the encoder is in the Acknowledge (Ack) state. The host should then switch to its receive state.
Data Reception: The host synchronously receives data transmitted by the encoder over the SLO line, synchronized with the MA clock signal. The data frame format is Most Significant Bit (MSB) first.
Cycle Completion: After receiving all data, the host pulls the MA clock line high. The encoder will then pull its SLO line high to indicate readiness for the next position request. If not ready, it will keep SLO low.

4.2.2 Software Module Design

The BISS module's functional structure comprises three main sub-modules: the MA module, the SLO module, and the CRC module. Key signals for the BISS module include:

biss_en: The module enable signal, used to initiate the process of acquiring encoder position information.
data_length: A configurable parameter that sets the bit width of the encoder's position data.
SLO: The BISS protocol data line, carrying information from the encoder to the FPGA.
MA: The BISS protocol clock line, generated by the FPGA to synchronize communication with the encoder.
abs_data: The single-turn absolute value, used for multi-turn counting in rotary encoders and for electrical angle calculations.
abs_pos: The total actual position data from the encoder.

The module operates by first configuring the data_length register in the FPGA to match the position data bit width of the specific encoder model being used. This design supports position data widths up to 32 bits. When the main control module asserts the biss_en signal, the MA clock module begins operation, sending a fixed-frequency MA clock signal to the encoder to request its current position value. Upon receiving the data returned by the encoder, the FPGA feeds this data into the CRC verification module for integrity checking. Finally, the validated position data is output to the next stage of the control system.

To manage the data reception process from the SLO line, a state machine is implemented, dividing the reception into six distinct states based on the BISS protocol's timing diagram characteristics:

Idle State: The functional module starts in an idle state.
Wait State: It then transitions into a waiting state, monitoring the SLO line.
Ack State: When the encoder pulls the SLO signal line low, the state machine enters the Acknowledge (Ack) state.
STATE_ZERO: Upon detecting SLO transitioning from low to high, the state machine moves to STATE_ZERO.
Work State: When the SLO data line is detected as low again, the module enters the working state and begins receiving data.
TIMEOUT State: After receiving the configured data width, the module transitions to the "TIMEOUT" state, indicating the end of data reception for the current cycle.

In practical engineering applications, data signal levels can be susceptible to glitches or metastability, especially when sampling on the rising edge of a clock signal. Directly reading the SLO data line's level on the MA clock's rising edge can lead to unreliable data. A more robust approach, implemented here, is to sample the SLO data line multiple times during the stable middle portion of each MA clock cycle. The logic then determines which level (high or low) has a majority count among these samples and assigns that stable value to a register. This method leverages the fact that the MA clock frequency is typically much lower than the internal FPGA operating frequency (sys_clk), allowing for multiple samples within a single MA clock period. Specifically, at each rising edge of the MA clock, a counter (bps_cnt) starts from zero, accumulating samples. The system then compares the count of high samples versus low samples, and the majority value is latched into a register. This process repeats until all data bits are received.

Once the SLO module completes data reception, it sends a "reception complete" signal to both the MA module and the CRC verification module. The MA module, upon receiving this signal, instructs the FPGA to pull the MA clock line high, preparing for the next communication cycle. Simultaneously, the CRC module receives the data frame transmitted by the SLO module and performs the CRC checksum calculation. The BISS-C protocol specifies a CRC polynomial of G(x) = x^6 + x + 1, which corresponds to the binary sequence 1000011. To maximize the parallel processing capabilities of the FPGA and enhance efficiency, a parallel CRC calculation algorithm is employed. This allows the CRC checksum to be computed within a single system clock cycle. The core logic for this parallel CRC operation involves an input d[33:0] (the data to be checked) and an output crc_out[5:0] (the calculated CRC result), with an initial c[5:0] value of zero.

After calculation, the crc_out result is compared with the received CRC checksum from the encoder. A crucial detail is that the encoder internally inverts its CRC checksum before transmitting it to the host. Therefore, the FPGA's calculated CRC value must also be inverted before comparison. If the inverted calculated CRC matches the received CRC, it indicates that the SLO module correctly received the data, and the validated position data is then updated in the next-level module. If there is a mismatch, the system retains the previously valid position data and initiates a new position request to the encoder.

4.2.3 Simulation Test

A comprehensive test file was developed to verify the BISS module's functionality using ModelSim software, with a strong emphasis on stability for real-world industrial applications. The simulation was conducted at a communication rate of 5Mbps, which is a common high-speed setting. The module was tested against frequently used encoder models with both 26-bit and 32-bit position data widths.

For the 26-bit data width, two sets of data were configured. The first set intentionally had a single bit difference in the last bit of the CRC checksum, while the second set was entirely correct. This deliberate manipulation allowed for testing whether the BISS module could correctly receive and differentiate between valid and corrupted data, and critically, to verify the proper operation of the CRC verification module. As noted previously, the BISS-C protocol specifies that the encoder inverts its CRC checksum before transmission. Consequently, the CRC values presented in the test data tables were pre-inverted. During the FPGA's verification process, the calculated CRC value must also be inverted before being compared with the received value. The overall simulation results for the 26-bit mode confirmed the module's correct operation.

The simulation waveforms showed that upon receiving the biss_en enable signal, the FPGA transmitted a 5MHz MA clock signal to the encoder. The encoder, detecting the MA clock, returned data via the SLO line. Once the FPGA completed data reception, it transferred the received CRC checksum (crc_check) and the data to be verified (check_data) to the CRC module, along with a done signal to trigger the CRC module's operation. The CRC module then processed the data. After processing, it compared its calculated CRC (crc_out) with the received crc_check. If they matched (after accounting for the inversion), a high-level pulse on the crc_done signal indicated successful verification; otherwise, crc_done remained low.

A closer examination of the CRC verification details revealed that if the received crc_check was, for example, 001011, and the calculated crc_out (before inversion) was 110100, then after inverting crc_out to 001011, it matched the received value. This confirmed the CRC module's correct operation, and importantly, the CRC calculation was completed within a single system clock cycle, demonstrating the efficiency of the parallel CRC algorithm.

From these simulation results, it was concluded that the BISS module could reliably send MA clock signals at 5Mbps, completely receive encoder data, and perform CRC verification within a single clock cycle. Thus, the designed BISS module successfully passed functional simulation tests for the 26-bit mode.

Similarly, simulation tests were performed for the 32-bit data width, using a corresponding set of simulation data. The functional simulation and CRC verification details for the 32-bit mode also confirmed the successful operation of the designed BISS module.

In conclusion, the designed BISS module is capable of reliably receiving data from encoders with data widths up to 32 bits at a communication rate of 5Mbps, simply by adjusting the data_length register.

4.3 Tamagawa Protocol Program Design

The Tamagawa protocol, developed by Tamagawa Seiki Co., Ltd. of Japan, employs a "request-response" communication model. In this scheme, the host (FPGA) sends various request commands to the encoder to obtain different types of data. The communication method is similar to standard serial communication, transmitting one byte of data at a fixed rate of 2.5Mbps, starting with the least significant bit (LSB). This section will primarily focus on the Tamagawa protocol's mode for reading encoder information.

4.3.1 Tamagawa Protocol Data Structure

The data frame format for reading encoder information consists of several components. The operational principle is that the host sends different control frame commands to the encoder to request specific information. In response, the encoder returns a data group comprising a control frame, a status frame, a variable number of data frames (depending on the encoder model and requested information), and a CRC frame for error checking.

The control frame's data format is structured into three main parts: a synchronization code, a control command, and a control command checksum bit. The synchronization code is a fixed, unchanging pattern, while the control command and its associated checksum bit are specific instruction codes. These instruction codes, detailed in a command table, correspond to different functions and dictate the type of data the encoder will return. For instance, if a user wishes to obtain both the single-turn and multi-turn values from the encoder, consulting the command table would reveal that instruction code ID 3 (binary: 01011000) corresponds to this request. The host would then send this specific command code to the encoder. Upon receiving it, the encoder would respond with its current single-turn value, multi-turn value, and any additional relevant information.

The status frame's data format provides critical diagnostic information and is composed of information bits, encoder error alarm bits, and communication alarm bits. The meaning of the information bits can vary depending on the specific encoder model and should be referenced in the corresponding encoder manual. The encoder error alarm bits include ea0 and ea1. If ea0 is set to '1', it indicates an encoder encoding error or an external power supply issue, signifying that the current encoder information is incorrect. If ea1 is '1', it indicates an error in the encoder's multi-turn value data. The communication alarm bits, ca0 and ca1, signal issues with the communication itself. If ca0 is '1', it means an error occurred in the control command checksum bit within the received control frame. If ca1 is '1', it indicates an error in the end bit of the received control frame.

Tamagawa absolute encoders are capable of achieving high-resolution position encoding data, up to 39 bits in total, which typically includes 23 bits for the single-turn value and 16 bits for the multi-turn value. However, the Tamagawa protocol transmits data only 8 bits at a time. Therefore, sending complete high-resolution position information requires multiple communication exchanges.

Difference Between .c and .h Files (The Relationship Between Header Files and Implementation Files)

Leo Liu — Thu, 21 May 2026 07:14:38 +0000

Originally published on the Sienovo Engineering Blog. Sienovo is the overseas brand of 深圳信迈 (Shenzhen Xinmai), building edge AI computing solutions for industrial video analytics.

Difference Between .c and .h Files

A simple question: What is the difference between .c and .h files?

After studying C for several months, I feel more and more confused. Subroutines can be defined either in a .c file or in a .h file — so what exactly is the difference in usage between these two?

Reply #2:

Do not define subroutines in .h files.

Function definitions should be placed in .c files, while .h files should only contain declarations. Otherwise, if included multiple times, it will lead to errors due to duplicate function definitions.

Reply #3:

.h files are for declarations only and do not generate code after compilation.

Reply #4:

The purpose is to achieve software modularity, making the software structure clear and also easier for others to use your code.

From a pure C language syntax perspective, you can technically place anything in a .h file, because #include is completely equivalent to copying and pasting the content of the .h file directly into the .c file.

.h files should contain macros, variable and function declarations — they tell others "what your program can do and how to use it."

.c files contain the actual definitions of variables and functions — they tell the computer "how your program is implemented."

Reply #5:

Of course, if a .h file is included by multiple .c files

and the .h file contains definitions of entities (variables or functions), duplicate definition errors will occur.

Declarations can appear any number of times, but definitions must be unique.

Reply #6:

Generally speaking, a C file should represent a module.

If your program consists of only one module (a single C file), you may not need a header file at all.

Otherwise, your module is clearly not independent and its implementation needs to be called by other modules. In this case, you should create a header file (H file) to declare which functions are public. Once other modules include your header file, they can use these public declarations.

Reply #7:

One C file corresponds to one H file — this makes management easier.

For example, if you have a "feed_dog.c", you should also create a "feed_dog.h":

ifndef _feed_dog_h

define _feed_dog_h

extern void feed_dog(void);

endif

Actually, it doesn't matter if you write function bodies in H files — it's just not conventional. As long as you follow the above format, you can include the H file as many times as needed, hehe.

Reply #8:

It's just a convention.

The compiler itself makes no distinction between .c and .h files. How you use .c and .h files is entirely up to the programmer. However, to ensure your code remains readable (to yourself and others) in the future, please follow common conventions — which have already been well explained by others above.

It's like driving on the right side of the road: it's a human-defined rule. The car (compiler) itself doesn't know whether it's driving on the left or right.

If you prefer, you could even use arbitrary extensions for source and header files, but doing so might cause your integrated development and debugging environment to fail, forcing you to write your own makefile.

Reply #9:

Thank you all very much, but I'm getting even more confused now:

When a function is used frequently (e.g., by a dozen C files), I usually put it in the H file and prefix it with __inline. For __inline functions, many C files can include this H file, but it seems only one H file can include it — if two H files include it, compilation errors occur.
Some array variables can be as large as十几K (tens of kilobytes) and require initial values, so I don't put them in C files — otherwise, it becomes too confusing.
#ifndef _feed_dog_h #define _feed_dog_h

extern void feed_dog(void);

endif

Brother Mohanwei, does this mean that feed_dog.h can be included an unlimited number of times?

Reply #11:

ifndef _feed_dog_h // If the macro "_feed_dog_h" has not been defined yet

define _feed_dog_h // Then define the macro "_feed_dog_h"

extern void feed_dog(void); // Declare an external function

endif // End of "#ifndef"

Therefore, no matter how many times you include it (even multiple times in the same C file), conflicts will not occur.

An article found online about .H and .C files — quite helpful, sharing it with everyone.

In simple terms:

To understand the difference between C files and header files, you first need to understand how a compiler works. Generally, the compiler performs the following steps:

Preprocessing phase
Lexical and syntactic analysis phase
Compilation phase: first convert to pure assembly code, then assemble into CPU-specific binary code, generating individual object files
Linking phase: perform absolute address relocation on code segments from various object files to generate a platform-specific executable file. Optionally, objcopy can be used to generate pure binary code, stripping away file format information.

The compiler works on a per-C-file basis. This means if your project contains no C files at all, it cannot be compiled. The linker works on object files, relocating functions and variables across one or more object files to generate the final executable. In PC-based software development, there is usually a main function, which is a convention across compilers. Of course, if you write your own linker script, you don't have to use main as the entry point!

With this background, let's return to the topic. To generate a final executable, you need some object files — hence you need C files. Among these C files, one must contain the main function as the program entry point. Let's start with a single C file. Suppose its content is:

include

include "mytest.h"

int main(int argc,char **argv)

{

test = 25;

printf("test.................%d\n",test);

}

And the header file content is:

int test;

Now, let's walk through how the compiler processes this example:

Preprocessing phase: The compiler treats each C file as a unit. It reads the C file and finds the first two lines include header files. It searches all include paths for these files. Once found, it processes macros, variables, function declarations, and nested includes in the headers, checks dependencies, performs macro substitution, and detects duplicate definitions or declarations. Finally, it effectively merges all content from the included files into the current C file, forming an intermediate "C file."
Compilation phase: In the previous step, the test variable from the header file is effectively merged into the intermediate C file, making test a global variable in this file. The compiler then allocates memory for all variables and functions, compiles functions into binary code, and generates an object file in a specific format. This object file contains symbol descriptions for global variables and functions, and organizes the binary code according to the target file standard.
Linking phase: The linker takes the object files generated in the previous step and, based on certain parameters, combines them into the final executable. Its main task is to relocate functions and variables across object files — essentially merging their binary code into a single file according to specific rules.

Now, returning to the question of what should go in C files versus header files:

Theoretically, anything supported by the C language can go in either .c or .h files. For example, you can write a function body in a header file. As long as some C file includes this header, the function will be compiled as part of that object file (since compilation is per C file, if no C file includes the header, the code is effectively dead). You can also place function declarations, variable declarations, or struct declarations in C files — no problem! So why do we separate code into header and C files? And why do we generally place function and variable declarations, macros, and struct declarations in headers, while definitions and implementations go in C files? The reasons are:

If a function body is implemented in a header file and that header is included by multiple C files, each C file that includes it will generate a copy of the function in its object file. If the function is not declared static (local), the linker will encounter multiple identical functions and report an error.
If a global variable is defined in a header file and initialized, then every C file that includes this header will have a copy of this variable. Since it has an initial value, the compiler places it in the DATA segment. During linking, multiple instances of the same variable will exist in the DATA segment, and the linker cannot merge them into a single variable (i.e., allocate only one storage space). However, if the variable is not initialized, the compiler places it in the BSS segment, and the linker will merge multiple instances of the same variable in BSS into a single storage location.
If macros, structs, or functions are declared in a C file, then to use them in another C file, you must repeat the declaration. If you modify a declaration in one C file but forget to update others, serious bugs can occur, making program logic unpredictable. By placing these common elements in a single header file, any C file that needs them can simply include the header — much more convenient. When you need to change a declaration, you only need to modify the header file.
Declaring structs, functions, etc., in header files allows you to package your code into a library for others to use without revealing the source code. How can others use your library functions? One way is to publish the source code, but another is to provide only the header file. Users can see your function prototypes in the header and know how to call them — just like using printf. How do you know its parameters? By looking at the declarations in its header file! Of course, these have become part of the C standard, so even without checking headers, you may already know how to use them.

What is the difference between ".h" and ".c" files in source code?

In a source code project, I see both udp.h and udp.c files — what's the relationship between them? What's the difference? Any experts care to help? Thanks!

Best answer: .c files are C source files, stored in text format, while .h files are header files — they contain function and global variable declarations in C. Since C functions are encapsulated, their implementation code is not visible.

The Relationship Between Header Files and Implementation Files

Today I came across an online article explaining .h and .c (.cpp) files. After reading it, I found some parts misleading, so I'd like to provide some guidance based on my understanding — especially for beginners.

Do you understand the basic meaning?

To understand the relationship between the two, we need to go back many years — long long ago, once upon a time...

It was a forgotten era when compilers only recognized .c (.cpp) files and had no idea what a .h file was.

People wrote many .c (.cpp) files and gradually noticed that the same declaration statements were repeated across many files. Yet they had to painstakingly retype them in every .c (.cpp) file. Worse, when a declaration changed, they had to manually search and update every file — a true apocalypse!

Finally, someone (or some people) could no longer endure this and extracted the repeated parts into a new file. Then, in any .c (.cpp) file that needed them, they added #include XXXX. Now, when a declaration changed, they only needed to update one file — and peace was restored!

Because this new file was typically placed at the top of .c (.cpp) files, it was called a "header file," with the extension .h.

From then on, compilers (actually preprocessors) learned that besides .c (.cpp) files, there were also .h files and a #include directive.

Although many changes have occurred since, this practice continues to this day — though over time, people have largely forgotten its origins.

Now that we've mentioned header files, let's discuss their roles.

I recall Lin Rui's concise description from High-Quality C/C++ Programming:

(1) Use header files to access library functionality. In many cases, source code cannot or must not be disclosed to users. Providing only header files and binary libraries is sufficient. Users can call library functions based on the interface declarations in the header, without needing to know the implementation details. The compiler extracts the appropriate code from the library.

(2) Header files enhance type safety. If an interface is implemented or used in a way inconsistent with its declaration in the header, the compiler will flag an error. This simple rule greatly reduces debugging and error-fixing effort.

Preprocessing is the compiler's precursor, responsible for combining program modules stored in different files into a complete source program.

#include is merely a simple file inclusion preprocessor command — it inserts the content of the specified file at that location. Other than that, it has no other purpose (at least that's my understanding).

I fully agree with Brother Qiankun Yixiao's view — foundational concepts must be clearly understood.

I'll now expand on his example to clarify some confusing points.

Example:

//a.h

void foo();

//a.c

include "a.h" // My question: Is this line necessary or not?

void foo()

{

return;

}

//main.c

include "a.h"

int main(int argc, char *argv[])

{

foo();

　return 0;

}

For the above code, answer three questions:

Is the #include "a.h" in a.c redundant?

Why do we often see xx.c including its corresponding xx.h?
If a.c does not include it, will the compiler automatically bind the contents of the .h file with the同名 .c file?
I'll rephrase the third question: If a.c does not include the .h file, will the compiler automatically bind the contents of the .h file with the同名 .c file?

Here is Qiankun Yixiao's original explanation:

From the C compiler's perspective, .h and .c files are irrelevant — you could rename them to .txt or .doc with little consequence. In other words, there is no inherent connection between .h and .c files. .h files typically contain declarations of variables, arrays, and functions defined in the同名 .c file — declarations that need to be accessible outside the .c file. What's the purpose of these declarations? Simply to make it convenient for other code to reference them. The #include "xx.h" directive literally means "remove this line and insert the entire content of xx.h here." Since many places need these function declarations (every place that calls functions from xx.c must declare them beforehand), using #include "xx.h" saves many lines of code — letting the preprocessor do the replacement. In short, xx.h exists only to save keystrokes for places that need to declare functions from xx.c. Whether the file that includes this .h is a .h, a .c, or the同名 .c file, there is no necessary relationship.

You might say: "Wait — if I only want to call one function from xx.c, but I include the entire xx.h, doesn't that bring in many useless declarations?" Yes, indeed, it introduces some "garbage." But it saves you effort and keeps the code cleaner. You can't have both fish and bear's paws — that's the trade-off. Anyway, extra declarations (since .h files usually contain only declarations, not definitions — see my article "Crossing the Road, Look Left and Right") do no harm and don't affect compilation — so why not?

Now, revisiting the three questions above — are they easier to answer?

His answers:

Answer: 1. Not necessarily. In this example, it's clearly redundant. But if functions in .c need to call other functions in the same .c file, including the同名 .h at the top avoids issues with declaration and call order (C requires declarations before use, and including the同名 .h at the beginning of the .c file solves this). Many projects even adopt this as a coding standard to ensure clean, readable code.

Answered in 1.
No. Anyone asking this question lacks clear understanding — or is trying to confuse things. It's extremely annoying that many Chinese exams include such poor-quality questions, seemingly designed only to confuse students.

Over!

One key point must be clarified: compilers work on compilation units. A compilation unit consists of a .c file and all .h files it includes. Intuitively, it's one file. A project can contain many files, one of which is the entry point — typically main() (though it's possible to have no such function and still run the program — see my blog). Without an entry point, the compilation unit only generates an object file (.o on Unix, .obj on Windows).

This example contains two compilation units: a.c and main.c. During the compilation phase, each generates its own .o file independently, without interaction with other files.

The #include preprocessor directive is handled during the preprocessing phase — which occurs before actual compilation and is handled by a preprocessor.

.h and .c files aren't entirely "irrelevant" — discussing them without considering the compiler is meaningless. Going deeper — such as how the OS loads the file, PE format (ELF on Linux), etc. — the compiler must first recognize the file to compile it. That's the prerequisite. If you change the extension, will the compiler still recognize it? At a higher level, Brother XX's point is valid — he means that just because two files have the same name doesn't imply a technical relationship; names are arbitrary.

The connection between them, as I mentioned earlier, is historical and habitual. Who wants to remember dozens of different filenames? (Take me as an example — if a data table has more than 30 fields, my head spins. Now some tables have over a hundred fields. I really hope someone invents a better method to make our world a better place.)

Qiankun Yixiao's third question is very representative — it appears frequently online. Modern compilers are absolutely not that intelligent, nor is there any need for them to be. Let's now discuss the compiler's processing flow (this is likely where beginners have doubts — lack of understanding of how .h and .c (.cpp) files change during compilation).

Let me give a simple example:

//a.h

class A

{

public:

int f(int t);

};

//a.cpp

include "a.h"

int A::f(int t)

{

return t;

}

//main.cpp

include "a.h"

void main()

{

A a;

a.f(3);

}

During preprocessing, when the preprocessor sees #include "filename", it reads that file in. For example, when compiling main.cpp and encountering #include "a.h", it reads the content of a.h. It now knows there is a class A with a member function f that takes an int parameter and returns an int. Proceeding further, it understands A a — creating an object of class A on the stack. Then it sees a call to A's member function f with parameter 3. Since it knows f expects an integer, and 3 matches, it places 3 on the stack and generates a call instruction (typically a call). It doesn't know where f is implemented — it leaves that blank to be resolved at link time. It also knows f returns an int, so it may prepare for that (though in this example, we don't use the return value, so it may ignore it). Reaching the end, main.cpp is compiled, producing main.obj. Throughout this process, it never needs to know the content of a.cpp.

Similarly, the compiler compiles a.cpp, generating the f() function and producing a.obj.

Finally, the linker combines all .obj files generated from the project's .cpp files.

At this stage, it locates the actual address of the f(int) function and fills in the blank address in main.obj. The final executable, main.exe, is produced.

Clear now? If not, let me explain further. When we study compiler principles, we know compilers work in phases, transforming the source program from one representation to another. Typically: source -> lexer -> parser -> semantic analyzer -> intermediate code generator -> optimizer -> code generator -> target program.

Among these, two key components involved in most phases are the symbol table and error handler.

Ultimately, it's the symbol table that causes confusion. The symbol table is a data structure. A fundamental task of the compiler is to record identifiers used in the source and collect attribute information for each — such as storage location, type, scope (where it's valid), etc. Simply put, when the compiler sees a symbol declaration (e.g., a function name), it registers it in the symbol table, storing information like entry address, number of parameters, return type, etc. The linking phase mainly resolves symbol references across files — what we commonly call "dereferencing."

After all this, is it clearer?

Finally, quoting Brother XXX's concluding three points:

Understanding syntax and concepts can be easy or hard. Three tips:

Don't work blindly — take time to think, reflect, and read.
Read good books and ask knowledgeable people. Bad books and weak programmers give you wrong concepts and mislead you.
Diligence compensates for lack of talent — hard work brings results.

If you think .c and .h files differ only in name, your understanding is too shallow. Looking at language evolution from procedural to object-oriented, header files resemble classes in some ways. Header files offer better encapsulation — this is easy to see. Just open any standard C library .h file — the pattern is obvious. So I agree with Brother XXX that Qiankun Yixiao's view is superficial.

But from another perspective:

(As for compiler implementation, I'm not fully informed. But I believe)

//a.cpp

include "a.h"

int A::f(int t)

{

return t;

}

programs like this probably don't exist... hehe. So modern developers simplifying .h and .c usage is also influenced by history and era.

I'm not very bright, but after reading several times, I finally got it.

Now summarizing (please correct me if wrong):

Header files can pre-inform the compiler of necessary declarations, allowing smooth compilation even before actual definitions appear. The significance of header files lies in: a. Making programs concise and clear. b. Avoiding redundant declaration code.
There is no inherent connection between **.c and **.h files. This article is from CSDN Blog. Please credit the source: http://blog.csdn.net/bm1408/archive/2006/02/22/606382.aspx ---

Linux Socket Network Programming (Theoretical Analysis + Comprehensive Examples)

Leo Liu — Thu, 21 May 2026 07:14:05 +0000

Originally published on the Sienovo Engineering Blog. Sienovo is the overseas brand of 深圳信迈 (Shenzhen Xinmai), building edge AI computing solutions for industrial video analytics.

What is a Socket?

The Socket interface is the API for TCP/IP networks. It defines numerous functions or routines that developers can use to build applications over TCP/IP networks. To master Internet-based TCP/IP network programming, one must understand the Socket interface.

Socket interfaces were originally designed for Unix operating systems. If you are familiar with Unix I/O operations, understanding Sockets becomes much easier. Network socket data transmission is a special form of I/O, and a socket is essentially a file descriptor. Similar to opening a file, there is a function call socket() that returns an integer socket descriptor. Subsequent operations such as connection establishment and data transfer are performed using this descriptor. There are two commonly used socket types: stream sockets (SOCK_STREAM) and datagram sockets (SOCK_DGRAM). Stream sockets are connection-oriented and used with TCP services; datagram sockets are connectionless and correspond to UDP services.

Socket Creation

To create a socket, a program calls the socket() function, which returns a handle similar to a file descriptor. The function prototype is:

int socket(int domain, int type, int protocol);

The domain parameter specifies the protocol family, usually PF_INET, indicating the Internet protocol family (TCP/IP). The type parameter defines the socket type: SOCK_STREAM or SOCK_DGRAM. The Socket interface also defines raw sockets (SOCK_RAW), allowing programs to access lower-level protocols. The protocol parameter is typically set to 0. The socket() call returns an integer socket descriptor, which can be used in subsequent operations.

A socket descriptor is a pointer to an internal data structure, pointing to an entry in the descriptor table. When socket() is called, the system allocates storage space for a socket data structure. The system manages the descriptor table on behalf of the application.

A network connection between two programs consists of five pieces of information: communication protocol, local protocol address, local host port, remote host address, and remote protocol port. This information is stored within the socket data structure.

Socket Configuration

After obtaining a socket descriptor via socket(), the socket must be configured before network transmission can occur. Connection-oriented socket clients use the connect() function to store local and remote information in the socket structure. Unconnected socket clients and servers, as well as connection-oriented socket servers, use the bind() function to configure local information.

The bind() function associates a socket with a port on the local machine, after which the application can listen for service requests on that port. The function prototype is:

int bind(int sockfd, struct sockaddr *my_addr, int addrlen);

Here, sockfd is the socket descriptor returned by socket(), my_addr is a pointer to a sockaddr structure containing the local IP address and port number, and addrlen is typically set to sizeof(struct sockaddr).

The struct sockaddr type is used to store socket information:

struct sockaddr {
    unsigned short sa_family; /* Address family, AF_xxx */
    char sa_data[14];         /* 14-byte protocol address */
};

The sa_family field is usually AF_INET, representing the Internet (TCP/IP) address family. The sa_data field contains the socket's IP address and port number.

Another commonly used structure is:

struct sockaddr_in {
    short int sin_family;           /* Address family */
    unsigned short int sin_port;    /* Port number */
    struct in_addr sin_addr;        /* IP address */
    unsigned char sin_zero[8];      /* Padding to match struct sockaddr size */
};

This structure is more convenient. The sin_zero field ensures sockaddr_in is the same size as sockaddr and can be zeroed using bzero() or memset(). Pointers to sockaddr_in and sockaddr can be cast interchangeably, meaning you can pass a sockaddr_in* where a sockaddr* is expected, and vice versa.

When using bind(), you can use the following assignments to automatically obtain the local IP address and a randomly selected unused port:

my_addr.sin_port = 0;                    /* System selects an unused port */
my_addr.sin_addr.s_addr = INADDR_ANY;   /* Use local IP address */

Setting my_addr.sin_port to 0 lets the system automatically choose an available port. Setting my_addr.sin_addr.s_addr to INADDR_ANY tells the system to fill in the local IP address.

Note: When using bind(), sin_port and sin_addr must be converted to network byte order; sin_addr itself does not require conversion.

Computers use two byte orders: big-endian (most significant byte first) and little-endian (least significant byte first). Internet data is transmitted in big-endian order. Machines using little-endian internally must convert data before transmission to avoid inconsistencies.

The following functions perform byte order conversion:

htonl(): Convert a 32-bit value from host to network byte order
htons(): Convert a 16-bit value from host to network byte order
ntohl(): Convert a 32-bit value from network to host byte order
ntohs(): Convert a 16-bit value from network to host byte order bind() returns 0 on success and -1 on error, setting errno to the appropriate error code. Avoid using port numbers below 1024 when calling bind(), as ports 1–1024 are reserved. Choose any unused port above 1024.

Connection Establishment

Connection-oriented client programs use the connect() function to configure the socket and establish a TCP connection with a remote server. The function prototype is:

int connect(int sockfd, struct sockaddr *serv_addr, int addrlen);

sockfd is the socket descriptor returned by socket(); serv_addr is a pointer to a structure containing the remote host's IP address and port; addrlen is the size of the address structure. connect() returns -1 on error and sets errno accordingly. Client programs typically do not call bind() because they only need to know the destination IP address. The system automatically selects an unused local port and notifies the program when data arrives.

connect() initiates a direct connection to the remote host. Only connection-oriented clients need to connect their socket to a remote host. Connectionless protocols never establish direct connections. Connection-oriented servers do not initiate connections; they passively listen for client requests on a port.

The listen() function places the socket in passive listening mode and creates an incoming connection queue, storing service requests until the program processes them.

int listen(int sockfd, int backlog);

sockfd is the socket descriptor returned by socket(); backlog specifies the maximum number of pending connections in the queue, which wait for accept() (see below). Most systems default backlog to 20. If a new request arrives when the queue is full, the socket rejects the connection, and the client receives an error.

listen() returns -1 on error and sets errno.

The accept() function allows a server to accept incoming client connection requests. After setting up the input queue, the server calls accept(), then sleeps until a connection request arrives.

int accept(int sockfd, void *addr, int *addrlen);

sockfd is the listening socket descriptor; addr is typically a pointer to a sockaddr_in variable that stores information about the requesting host (which host, which port); addrlen is usually a pointer to an integer with value sizeof(struct sockaddr_in). On error, accept() returns -1 and sets errno.

When accept() detects a connection request on the monitored socket, the system creates a new socket, associates it with the requesting process's address, and returns its descriptor. The original socket continues listening, while data transmission occurs over the new socket.

Data Transmission

The send() and recv() functions are used for data transfer over connection-oriented sockets.

send() prototype:

int send(int sockfd, const void *msg, int len, int flags);

sockfd is the socket descriptor for data transmission; msg is a pointer to the data to send; len is the data length in bytes; flags is usually 0 (refer to man pages for advanced usage).

send() returns the number of bytes actually sent, which may be less than requested. Always compare the return value with len and handle partial sends appropriately.

Example:

char *msg = "Hello!";
int len, bytes_sent;
...
len = strlen(msg);
bytes_sent = send(sockfd, msg, len, 0);
...

recv() prototype:

int recv(int sockfd, void *buf, int len, unsigned int flags);

sockfd is the receiving socket descriptor; buf is the buffer to store received data; len is buffer size; flags is usually 0. recv() returns the number of bytes actually received, or -1 on error with errno set.

sendto() and recvfrom() are used for connectionless datagram sockets. Since no connection is established, the destination address must be specified during data transmission.

sendto() prototype:

int sendto(int sockfd, const void *msg, int len, unsigned int flags, const struct sockaddr *to, int tolen);

This function adds two parameters: to, which contains the destination IP and port, and tolen, typically sizeof(struct sockaddr). sendto() returns the number of bytes sent or -1 on error.

recvfrom() prototype:

int recvfrom(int sockfd, void *buf, int len, unsigned int flags, struct sockaddr *from, int *fromlen);

from is a sockaddr variable storing the source IP and port. fromlen is usually sizeof(struct sockaddr). On return, fromlen contains the actual size of data stored in from. recvfrom() returns the number of bytes received or -1 on error with errno set.

If connect() is called on a datagram socket, send() and recv() can be used for data transfer. The socket remains a datagram socket using UDP, but the kernel automatically adds source and destination address information during transmission.

Ending Transmission

After all data operations are complete, call close() to release the socket and stop all data operations:

close(sockfd);

Alternatively, use shutdown() to close the socket partially. This function allows stopping data transmission in one direction while continuing in the other. For example, you can disable sending while still receiving data until all incoming data is read.

int shutdown(int sockfd, int how);

sockfd is the socket descriptor to close. The how parameter specifies the shutdown mode:

0 — Disallow further receives
1 — Disallow further sends
2 — Disallow both sends and receives To fully close, call close(). shutdown() returns 0 on success and -1 on error with errno set.

Socket Programming Example

The server in this example sends the string "Hello, you are connected!" to the client via a socket connection. Running the server software on the server and the client software on the client will result in the client receiving the string.

Server code:

#include <stdio.h>
#include <stdlib.h>
#include <errno.h>
#include <string.h>
#include <sys/types.h>
#include <netinet/in.h>
#include <sys/socket.h>
#include <sys/wait.h>
#define SERVPORT 3333 /* Server listening port */
#define BACKLOG 10     /* Max concurrent connection requests */
main()
{
    int sockfd, client_fd; /* sockfd: listening socket; client_fd: data transfer socket */
    struct sockaddr_in my_addr;    /* Local address info */
    struct sockaddr_in remote_addr; /* Client address info */
    int sin_size;
    if ((sockfd = socket(AF_INET, SOCK_STREAM, 0)) == -1) {
        perror("socket creation failed!");
        exit(1);
    }
    my_addr.sin_family = AF_INET;
    my_addr.sin_port = htons(SERVPORT);
    my_addr.sin_addr.s_addr = INADDR_ANY;
    bzero(&(my_addr.sin_zero), 8);
    if (bind(sockfd, (struct sockaddr *)&my_addr, sizeof(struct sockaddr)) == -1) {
        perror("bind failed!");
        exit(1);
    }
    if (listen(sockfd, BACKLOG) == -1) {
        perror("listen failed!");
        exit(1);
    }
    while(1) {
        sin_size = sizeof(struct sockaddr_in);
        if ((client_fd = accept(sockfd, (struct sockaddr *)&remote_addr, &sin_size)) == -1) {
            perror("accept failed");
            continue;
        }
        printf("received a connection from %s\n", inet_ntoa(remote_addr.sin_addr));
        if (!fork()) { /* Child process code */
            if (send(client_fd, "Hello, you are connected!\n", 26, 0) == -1)
                perror("send failed!");
            close(client_fd);
            exit(0);
        }
        close(client_fd);
    }
}

The server workflow: First, socket() creates a socket. Then bind() associates it with the local address and port. listen() starts listening on that socket. When accept() receives a connection request, it creates a new socket. The server prints the client's IP address and sends the greeting via the new socket, then closes it.

The fork() function creates a child process to handle data transfer. fork() returns 0 in the child process, so the if block contains child process code, executed concurrently with the parent process code that follows.

Client code:

#include<stdio.h>
#include <stdlib.h>
#include <errno.h>
#include <string.h>
#include <netdb.h>
#include <sys/types.h>
#include <netinet/in.h>
#include <sys/socket.h>
#define SERVPORT 3333
#define MAXDATASIZE 100 /* Max data size per transfer */
main(int argc, char *argv[]){
    int sockfd, recvbytes;
    char buf[MAXDATASIZE];
    struct hostent *host;
    struct sockaddr_in serv_addr;
    if (argc < 2) {
        fprintf(stderr,"Please enter the server's hostname!\n");
        exit(1);
    }
    if((host=gethostbyname(argv[1]))==NULL) {
        herror("gethostbyname failed!");
        exit(1);
    }
    if ((sockfd = socket(AF_INET, SOCK_STREAM, 0)) == -1){
        perror("socket creation failed!");
        exit(1);
    }
    serv_addr.sin_family=AF_INET;
    serv_addr.sin_port=htons(SERVPORT);
    serv_addr.sin_addr = *((struct in_addr *)host->h_addr);
    bzero(&(serv_addr.sin_zero),8);
    if (connect(sockfd, (struct sockaddr *)&serv_addr, sizeof(struct sockaddr)) == -1) {
        perror("connect failed!");
        exit(1);
    }
    if ((recvbytes=recv(sockfd, buf, MAXDATASIZE, 0)) == -1) {
        perror("recv failed!");
        exit(1);
    }
    buf[recvbytes] = '\0';
    printf("Received: %s",buf);
    close(sockfd);
}

The client first resolves the server's hostname to an IP address using gethostbyname(), then creates a socket and connects to the server. After a successful connection, it receives data and closes the socket.

gethostbyname() performs domain name resolution. Since IP addresses are hard to remember, domain names are commonly used, requiring conversion. Function prototype:

struct hostent *gethostbyname(const char *name);

It returns a pointer to a hostent structure:

struct hostent {
    char *h_name;          /* Official host name */
    char **h_aliases;      /* NULL-terminated array of host aliases */
    int h_addrtype;        /* Address type, AF_INET in Internet */
    int h_length;          /* Length of address in bytes */
    char **h_addr_list;    /* NULL-terminated array of host addresses */
};
#define h_addr h_addr_list[0] /* First address in h_addr_list */

On success, gethostbyname() returns a pointer to struct hostent; on failure, it returns NULL. Use herror() instead of perror() to print error messages.

Connectionless client/server programs work similarly in principle to connection-oriented ones. The key difference is that connectionless clients typically do not establish a connection and must specify the remote address when sending or receiving data.

Blocking and Non-blocking

A blocking function prevents the program from calling another function until its task is complete. For example, when a program executes a read operation, it waits until the read finishes before proceeding. When a server reaches the accept() statement and no client connection request has arrived, it blocks, waiting for a request. This is called blocking. Non-blocking operations return immediately. For instance, if a server should only check for pending connections without waiting, the socket can be set to non-blocking mode. In non-blocking mode, accept() returns immediately if no client is waiting.

#include <unistd.h>
#include <fcntl.h>
...
sockfd = socket(AF_INET, SOCK_STREAM, 0);
fcntl(sockfd, F_SETFL, O_NONBLOCK);
...

Setting a socket to non-blocking enables polling of multiple sockets. Attempting to read from a non-blocking socket with no data returns immediately with -1 and sets errno to EWOULDBLOCK. However, polling wastes CPU cycles. The select() system call solves this efficiently by suspending the process while the kernel monitors a set of file descriptors. When activity occurs on any monitored descriptor, select() returns, indicating which descriptor is ready. This avoids wasteful CPU polling. The select() prototype is:

int select(int numfds, fd_set *readfds, fd_set *writefds,
           fd_set *exceptfds, struct timeval *timeout);

readfds, writefds, and exceptfds are sets of file descriptors monitored for read, write, and exception events. To check if data can be read from standard input and a socket, add file descriptor 0 and the socket descriptor to readfds. numfds is the highest-numbered file descriptor plus one—in this case, sockfd + 1. After select() returns, readfds is modified to indicate ready descriptors. Use FD_ISSET() to test. Macros for manipulating fd_set:

FD_ZERO(fd_set *set) — Clear the set
FD_SET(int fd, fd_set *set) — Add a descriptor to the set
FD_CLR(int fd, fd_set *set) — Remove a descriptor from the set
FD_ISSET(int fd, fd_set *set) — Test if a descriptor is set The timeout parameter is a pointer to a struct timeval, allowing select() to return after the specified time if no descriptors are ready. The structure:

struct timeval {
    int tv_sec;    /* seconds */
    int tv_usec;   /* microseconds */
};

POP3 Client Example

This example implements a POP3 client that connects to a mail server and retrieves emails for a specified account. Commands are stored in the POPMessage array and sent in a do-while loop.

#include<stdio.h>
#include <stdlib.h>
#include <errno.h>
#include <string.h>
#include <netdb.h>
#include <sys/types.h>
#include <netinet/in.h>
#include <sys/socket.h>
#define POP3SERVPORT 110
#define MAXDATASIZE 4096

main(int argc, char *argv[]){
    int sockfd;
    struct hostent *host;
    struct sockaddr_in serv_addr;
    char *POPMessage[]={
        "USER userid\r\n",
        "PASS password\r\n",
        "STAT\r\n",
        "LIST\r\n",
        "RETR 1\r\n",
        "DELE 1\r\n",
        "QUIT\r\n",
        NULL
    };
    int iLength;
    int iMsg=0;
    int iEnd=0;
    char buf[MAXDATASIZE];

    if((host=gethostbyname("your.server"))==NULL) {
        perror("gethostbyname error");
        exit(1);
    }
    if ((sockfd = socket(AF_INET, SOCK_STREAM, 0)) == -1){
        perror("socket error");
        exit(1);
    }
    serv_addr.sin_family=AF_INET;
    serv_addr.sin_port=htons(POP3SERVPORT);
    serv_addr.sin_addr = *((struct in_addr *)host->h_addr);
    bzero(&(serv_addr.sin_zero),8);
    if (connect(sockfd, (struct sockaddr *)&serv_addr,sizeof(struct sockaddr))==-1){
        perror("connect error");
        exit(1);
    }

    do {
        send(sockfd,POPMessage[iMsg],strlen(POPMessage[iMsg]),0);
        printf("have sent: %s",POPMessage[iMsg]);

        iLength=recv(sockfd,buf+iEnd,sizeof(buf)-iEnd,0);
        iEnd+=iLength;
        buf[iEnd]='\0';
        printf("received: %s,%d\n",buf,iMsg);

        iMsg++;
    } while (POPMessage[iMsg]);

    close(sockfd);
}

A robust Linux/Unix network file transfer example:

I've been studying Unix network programming recently. Initially, when transferring image files, extra bytes would appear. After testing and reviewing online articles, I discovered the issue was improper handling of the final read/write operations. I've improved the program, and it now successfully transfers various file types. Next step: implement video file transfer.

server.c

#include <sys/types.h>
#include <sys/socket.h>
#include <stdio.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <unistd.h>

#define MAXSIZE 100

int main(int argc,char **argv)
{
    int server_sockfd,client_sockfd;
    int server_len,client_len;
    char ch[MAXSIZE];
    struct sockaddr_in server_address;
    struct sockaddr_in client_address;

    if (argc != 2)
    {
        printf("server:file name\n");
        exit(1);
    }

    server_sockfd = socket(AF_INET,SOCK_STREAM,0);
    if(server_sockfd <0)
    {
        printf("Creating socket error!\n");
        exit(1);
    }

    bzero(&server_address,sizeof(struct sockaddr_in));
    server_address.sin_family = AF_INET;
    server_address.sin_addr.s_addr = htonl(INADDR_ANY);
    server_address.sin_port = htons(5000);
    server_len = sizeof(server_address);

    int opt = 1;
    setsockopt(server_sockfd,SOL_SOCKET,SO_REUSEADDR,&opt,sizeof(opt));

    if(bind(server_sockfd,(struct sockaddr *)&server_address,server_len) < 0)
    {
        perror("Bind error.");
        exit(1);
    }

    if(listen(server_sockfd,5) == -1)
    {
        printf("listen error!\n");
        exit(1);
    }

    client_len = sizeof(client_address);
    client_sockfd = accept(server_sockfd,(struct sockaddr *)&client_address, &client_len);
    if(client_sockfd == -1)
    {
        printf("accept error!\n");
        exit(1);
    }

    FILE *fd = fopen(argv[1],"rb");
    if (fd == NULL)
    {
        printf("file open error!\n");
        exit(2);
    }
    while(!feof(fd))
    {
        int len = fread(ch,1,MAXSIZE,fd);
        write(client_sockfd,ch,len);
    }

    close(client_sockfd);
    fclose(fd);
    return 0;
}

client.c

#include <sys/socket.h>
#include <sys/types.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <unistd.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <netdb.h>
#define MAXSIZE 100

int main(int argc,char **argv)
{
    int sockfd,len;
    int result,count;
    char recvs[MAXSIZE];
    struct sockaddr_in address;
    struct hostent *host;

    if (argc != 3)
    {
        printf("client ip filename\n");
        exit(1);
    }

    host = gethostbyname(argv[1]) ;
    if(( sockfd = socket(AF_INET,SOCK_STREAM,0))== -1)
    {
        printf("socket create error!\n");
        exit(1);
    }

    bzero(&address,sizeof(address));
    address.sin_family = AF_INET;
    address.sin_addr = *((struct in_addr *)host->h_addr);
    address.sin_port = htons(5000);

    int opt = 1;
    setsockopt(sockfd,SOL_SOCKET,SO_REUSEADDR,&opt,sizeof(opt));

    len = sizeof(address);
    result = connect(sockfd,(struct sockaddr *)&address,len);
    if (result == -1)
    {
        perror("client");
        exit(1);
    }

    FILE *fd = fopen(argv[2],"wb");
    if (fd == NULL)
    {
        printf("create file error!\n");
        exit(1);
    }

    while(1){
        count = read(sockfd,recvs,MAXSIZE);
        if (count ==0)
            break;

        fwrite(recvs,1,count,fd);
    }

    close(sockfd);
    fclose(fd);
    return 0;
}

Socket, Ports, and Processes Q&A (Collected and Organized)

Leo Liu — Thu, 21 May 2026 07:14:02 +0000

Originally published on the Sienovo Engineering Blog. Sienovo is the overseas brand of 深圳信迈 (Shenzhen Xinmai), building edge AI computing solutions for industrial video analytics.

1. How many ports does a computer have, what are they, and what are their purposes?

Ports are divided into three main categories:

1) Well-known Ports (0–1023): These are tightly bound to specific services. Communications on these ports typically indicate a specific protocol. For example, port 80 is always used for HTTP traffic.

2) Registered Ports (1024–49151): These are loosely associated with specific services. Many services use these ports, and they may also serve other purposes. For instance, systems often begin assigning dynamic ports around 1024.

3) Dynamic and/or Private Ports (49152–65535): These should theoretically not be assigned to services. In practice, most systems assign dynamic ports starting from 1024, though exceptions exist—such as SUN's RPC ports, which start from 32768.

----------------------------------------------------------------------------------------------------------------------------------

2. A computer has only one IP address but multiple distinct ports

An IP address identifies a host; identical IP addresses usually indicate the same machine.

A port identifies a process. Two processes on the same machine cannot occupy the same port simultaneously, so if ports differ, they may belong to different processes on the same host.
Ports used for direct internet access typically start from 4000.
If two QQ instances are running on the same machine, their ports are usually 4000 and 4001, respectively.

--------------------------------------------------------------------------

3. Processes and Ports

1) Port Definition

3) Ports

A port is a named and addressable communication endpoint in a network, representing a resource that the operating system can allocate.

According to the OSI seven-layer model, the key functional difference between the transport layer and the network layer is that the transport layer provides process-to-process communication. Thus, the final destination in network communication is not just a host address, but also includes an identifier for a specific process. To address this, the TCP/IP protocol suite introduces the concept of a protocol port (commonly referred to as a "port") to identify communicating processes.

A port is an abstract software structure (including data structures and I/O buffers). After an application (i.e., a process) establishes a connection (binding) to a port via a system call, all data delivered by the transport layer to that port is received by the corresponding process, and all data sent by the process to the transport layer is transmitted through that port. In TCP/IP implementations, port operations resemble standard I/O operations—acquiring a port is akin to obtaining a unique local I/O file, which can be accessed using standard read/write primitives.

Similar to file descriptors, each port has an integer identifier called a port number to distinguish it from others. Since the TCP and UDP protocols in the TCP/IP transport layer are independent modules, their port number spaces are separate. For example, TCP can have port 255, and UDP can simultaneously have port 255 without conflict.

Port number allocation is critical and follows two basic methods:

Global allocation: A centralized approach where a recognized central authority assigns ports based on user needs and publishes the assignments.
Local allocation (dynamic allocation): A process requests a port from the local operating system, which returns a locally unique port number. The process then binds itself to this port via a system call.

TCP/IP combines both methods. It divides port numbers into two ranges: a small set of well-known ports assigned globally to standard services. Thus, every standard server has a universally recognized port number (e.g., port 80 for HTTP), consistent across machines. The remaining ephemeral ports are allocated locally. Both TCP and UDP reserve port numbers below 256 for well-known services.

Addressing

In network communication, the two communicating processes reside on different machines. In an interconnected network, machines may belong to different subnets, connected via network devices (gateways, bridges, routers, etc.). Hence, a three-level addressing scheme is required:

A host may connect to multiple networks, so a specific network address must be specified.
Each host on the network must have a unique address.
Each process on a host must have a unique identifier within that host.

Typically, a host address consists of a network ID and a host ID, represented as a 32-bit integer in TCP/IP. Both TCP and UDP use 16-bit port numbers to identify user processes.

Network Byte Order

Different computers store multi-byte values in different orders—some store the least significant byte at the lowest address (little-endian), others the most significant byte (big-endian). To ensure data integrity, network protocols define a standard byte order. TCP/IP uses big-endian (most significant byte first) for 16-bit and 32-bit integers, as defined in protocol header files.

Connection

The communication link between two processes is called a connection. Internally, a connection consists of buffers and protocol mechanisms; externally, it provides higher reliability than connectionless communication.

Half-association

As discussed, a process can be uniquely identified globally by a triple:

(Protocol, Local Address, Local Port Number)

This triple is known as a half-association, specifying one half of a connection.

Full Association

A complete inter-process communication requires two processes using the same high-level protocol—e.g., both ends must use TCP or both UDP. Therefore, a full communication is identified by a quintuple:

(Protocol, Local Address, Local Port, Remote Address, Remote Port)

This quintuple is called an association. Only two half-associations using the same protocol can form a valid association, fully specifying a connection.

------------------------------------------------------------------------------------------------

4. Revisiting Process-to-Port Mapping

Creation Date: 2002-02-05

Article Type: Original

Source: www.whitecell.org

Submitted by: ILSY (masteruser_at_263.net)

Author : ilsy

Email : ilsy@whitecell.org

HomePage: http://www.whitecell.org

Date : 2002-02-06

Category: Security

Keywords: process, PDE, PTE, paging, kernel object, linear address, physical address

There are many articles on process-to-port mapping. Here, I present my analysis of fport, explaining how it works.

fport.exe is a free tool developed by the Foundstone team that lists all open ports on a system and identifies which processes opened them.

The method described below is based on fport v1.33. If your version differs, please verify the version.

First, the tool checks whether the current user has administrative privileges (by reading the current process token). If not, it displays a warning and exits. Then, it adjusts the current process token. Next, it uses the ZwOpenSection function to open the kernel object \\Device\\PhysicalMemory, which allows access to physical system memory. The function prototype is as follows:

NTSYSAPI

NTSTATUS

NTAPI

ZwOpenSection(

OUT PHANDLE SectionHandle;

IN ACCESS_MASK DesiredAccess;

IN POBJECT_ATTRIBUTES ObjectAttributes

);

(See ntddk.h)

The first parameter receives the handle upon successful execution.

The second parameter, DesiredAccess, can be one of the following constants:

#define SECTION_QUERY 0x0001

#define SECTION_MAP_WRITE 0x0002

#define SECTION_MAP_READ 0x0004

#define SECTION_MAP_EXECUTE 0x0008

#define SECTION_EXTEND_SIZE 0x0010

#define SECTION_ALL_ACCESS (STANDARD_RIGHTS_REQUIRED|SECTION_QUERY|\\  
                        SECTION_MAP_WRITE |      \\  
                        SECTION_MAP_READ |       \\  
                        SECTION_MAP_EXECUTE |    \\  
                        SECTION_EXTEND_SIZE)

(See ntddk.h)

The third parameter is a structure containing information such as the object type to open:

typedef struct _OBJECT_ATTRIBUTES {

ULONG Length;

HANDLE RootDirectory;

PUNICODE_STRING ObjectName;

ULONG Attributes;

PVOID SecurityDescriptor; // Points to type SECURITY_DESCRIPTOR

PVOID SecurityQualityOfService; // Points to type SECURITY_QUALITY_OF_SERVICE

} OBJECT_ATTRIBUTES;

typedef OBJECT_ATTRIBUTES *POBJECT_ATTRIBUTES;

(See ntdef.h)

This structure is initialized using a macro:

#define InitializeObjectAttributes( p, n, a, r, s ) { \

(p)->Length = sizeof( OBJECT_ATTRIBUTES ); \

(p)->RootDirectory = r; \

(p)->Attributes = a; \

(p)->ObjectName = n; \

(p)->SecurityDescriptor = s; \

(p)->SecurityQualityOfService = NULL; \

}

(See ntdef.h)

To open the kernel object \\Device\\PhysicalMemory, the code is:

WCHAR PhysmemName[] = L"\\Device\\PhysicalMemory";

void * pMapPhysicalMemory;

HANDLE pHandle;

bool OpenPhysicalMemory()

{

NTSTATUS status;

UNICODE_STRING physmemString;

OBJECT_ATTRIBUTES attributes;

RtlInitUnicodeString( &physmemString, PhysmemName ); // Initialize Unicode string, see ntddk.h

InitializeObjectAttributes( &attributes, &physmemString,

OBJ_CASE_INSENSITIVE, NULL, NULL ); // Initialize OBJECT_ATTRIBUTES

status = ZwOpenSection(pHandle, SECTION_MAP_READ, &attributes ); // Open \Device\PhysicalMemory, get handle

if( !NT_SUCCESS( status ))

return false;

pMapPhysicalMemory = MapViewOfFile(pHandle, FILE_MAP_READ,

0, 0x30000, 0x1000);

// Map 0x1000 bytes starting from physical address 0x30000

if( GetLastError() != 0 )

return false;

return true;

}

Why start mapping from 0x30000? In Windows NT/2000, the system is divided into kernel mode (Ring 0) and user mode (Ring 3). All visible processes run in Ring 3. The physical address of the System process's Page Directory Entry (PDE) is typically 0x30000—the smallest PDE address in the system. The PDE contains 1024 entries, each pointing to a Page Table Entry (PTE). Each PTE manages 1024 pages of 4KB each. Since 1024 × 4KB = 0x1000 bytes, mapping 0x1000 bytes from 0x30000 covers one full PDE. (See WebCrazy's article "On Windows NT/2000 Paging Mechanism" for details.)

After opening \\Device\\PhysicalMemory, the program uses ZwOpenFile to open the kernel objects \\Device\\Tcp and \\Device\\Udp. The function prototype is:

NTSYSAPI

NTSTATUS

NTAPI

ZwOpenFile(

OUT PHANDLE FileHandle,

IN ACCESS_MASK DesiredAccess,

IN POBJECT_ATTRIBUTES ObjectAttributes,

OUT PIO_STATUS_BLOCK IoStatusBlock,

IN ULONG ShareAccess,

IN ULONG OpenOptions

);

(See ntddk.h)

The first parameter returns the handle to the opened object.

The second parameter, DesiredAccess, can be:

#define FILE_READ_DATA ( 0x0001 ) // file & pipe

#define FILE_LIST_DIRECTORY ( 0x0001 ) // directory

#define FILE_WRITE_DATA ( 0x0002 ) // file & pipe

#define FILE_ADD_FILE ( 0x0002 ) // directory

#define FILE_APPEND_DATA ( 0x0004 ) // file

#define FILE_ADD_SUBDIRECTORY ( 0x0004 ) // directory

#define FILE_CREATE_PIPE_INSTANCE ( 0x0004 ) // named pipe

#define FILE_READ_EA ( 0x0008 ) // file & directory

#define FILE_WRITE_EA ( 0x0010 ) // file & directory

#define FILE_EXECUTE ( 0x0020 ) // file

#define FILE_TRAVERSE ( 0x0020 ) // directory

#define FILE_DELETE_CHILD ( 0x0040 ) // directory

#define FILE_READ_ATTRIBUTES ( 0x0080 ) // all

#define FILE_WRITE_ATTRIBUTES ( 0x0100 ) // all

#define FILE_ALL_ACCESS (STANDARD_RIGHTS_REQUIRED | SYNCHRONIZE | 0x1FF)

#define FILE_GENERIC_READ (STANDARD_RIGHTS_READ |\

FILE_READ_DATA |\

FILE_READ_ATTRIBUTES |\

FILE_READ_EA |\

SYNCHRONIZE)

#define FILE_GENERIC_WRITE (STANDARD_RIGHTS_WRITE |\

FILE_WRITE_DATA |\

FILE_WRITE_ATTRIBUTES |\

FILE_WRITE_EA |\

FILE_APPEND_DATA |\

SYNCHRONIZE)

#define FILE_GENERIC_EXECUTE (STANDARD_RIGHTS_EXECUTE |\

FILE_READ_ATTRIBUTES |\

FILE_EXECUTE |\

SYNCHRONIZE)

(See ntdef.h)

The third parameter is the same structure as above.

The fourth parameter returns object attributes in an IO_STATUS_BLOCK structure:

typedef struct _IO_STATUS_BLOCK {

union {

NTSTATUS Status;

PVOID Pointer;

};

ULONG_PTR Information;

} IO_STATUS_BLOCK, *PIO_STATUS_BLOCK;

#if defined(_WIN64)

typedef struct _IO_STATUS_BLOCK32 {

NTSTATUS Status;

ULONG Information;

} IO_STATUS_BLOCK32, *PIO_STATUS_BLOCK32;

#endif

(See ntddk.h)

The fifth parameter, ShareAccess, can be:

#define FILE_SHARE_READ 0x00000001 // winnt

#define FILE_SHARE_WRITE 0x00000002 // winnt

#define FILE_SHARE_DELETE 0x00000004 // winnt

(See ntddk.h)

The sixth parameter, OpenOptions, can be:

#define FILE_DIRECTORY_FILE 0x00000001

#define FILE_WRITE_THROUGH 0x00000002

#define FILE_SEQUENTIAL_ONLY 0x00000004

#define FILE_NO_INTERMEDIATE_BUFFERING 0x00000008

#define FILE_SYNCHRONOUS_IO_ALERT 0x00000010

#define FILE_SYNCHRONOUS_IO_NONALERT 0x00000020

#define FILE_NON_DIRECTORY_FILE 0x00000040

#define FILE_CREATE_TREE_CONNECTION 0x00000080

#define FILE_COMPLETE_IF_OPLOCKED 0x00000100

#define FILE_NO_EA_KNOWLEDGE 0x00000200

#define FILE_OPEN_FOR_RECOVERY 0x00000400

#define FILE_RANDOM_ACCESS 0x00000800

#define FILE_DELETE_ON_CLOSE 0x00001000

#define FILE_OPEN_BY_FILE_ID 0x00002000

#define FILE_OPEN_FOR_BACKUP_INTENT 0x00004000

#define FILE_NO_COMPRESSION 0x00008000

#define FILE_RESERVE_OPFILTER 0x00100000

#define FILE_OPEN_REPARSE_POINT 0x00200000

#define FILE_OPEN_NO_RECALL 0x00400000

#define FILE_OPEN_FOR_FREE_SPACE_QUERY 0x00800000

#define FILE_COPY_STRUCTURED_STORAGE 0x00000041

#define FILE_STRUCTURED_STORAGE 0x00000441

#define FILE_VALID_OPTION_FLAGS 0x00ffffff

#define FILE_VALID_PIPE_OPTION_FLAGS 0x00000032

#define FILE_VALID_MAILSLOT_OPTION_FLAGS 0x00000032

#define FILE_VALID_SET_FLAGS 0x00000036

(See ntddk.h)

To open \\Device\\Tcp and \\Device\\Udp:

WCHAR physmemNameTcp[] = L"\\Device\\TCP";

WCHAR physmemNameUdp[] = L"\\Device\\UDP";

HANDLE pTcpHandle;

HANDLE pUdpHandle;

HANDLE OpenDeviceTcpUdp(WCHAR * deviceName)

{

NTSTATUS status;

UNICODE_STRING physmemString;

OBJECT_ATTRIBUTES attributes;

IO_STATUS_BLOCK iosb;

HANDLE pDeviceHandle;

RtlInitUnicodeString(&physmemString, deviceName);

if(GetLastError() != 0)

return NULL;

InitializeObjectAttributes( &attributes, &physmemString,

OBJ_CASE_INSENSITIVE, 0, NULL );

status = ZwOpenFile( &pDeviceHandle, 0x100000, &attributes, &iosb, 3, 0 );

if( !NT_SUCCESS( status ))

return NULL;

}

Next, the program uses ZwQuerySystemInformation to retrieve all handles and related information for currently running processes. The function prototype is:

NTSYSAPI

NTSTATUS

NTAPI

ZwQuerySystemInformation(

IN SYSTEM_INFORMATION_CLASS SystemInformationClass,

IN OUT PVOID SystemInformation,

IN ULONG SystemInformationLength,

OUT PULONG ReturnLength OPTIONAL

);

(This function structure is not publicly documented by Microsoft; see Gary Nebbett's "Windows NT/2000 Native API Reference".)

The first parameter is an enumeration specifying the system information type. ZwQuerySystemInformation supports 54 types; we use type 16, SystemHandleInformation.

The SYSTEM_HANDLE_INFORMATION structure is defined as:

typedef struct _SYSTEM_HANDLE_INFORMATION {

ULONG ProcessID; // Process identifier

UCHAR ObjectTypeNumber; // Object type

UCHAR Flags; // 0x01 = PROTECT_FROM_CLOSE, 0x02 = INHERIT

USHORT Handle; // Handle value

PVOID Object; // Kernel object address pointed to by handle

ACCESS_MASK GrantedAccess; // Access rights granted when handle was created

} SYSTEM_HANDLE_INFORMATION, *PSYSTEM_HANDLE_INFORMATION;

(Not publicly documented; see Nebbett's book.)

The second parameter outputs the query result.

The third sets the buffer size.

The fourth returns the required buffer size if the call fails due to insufficient buffer.

Code:

define SystemHandleInformation 16

PULONG GetHandleList()

{

ULONG cbBuffer = 0x1000;

PULONG pBuffer = new ULONG[cbBuffer];

NTSTATUS Status;

do {

Status = ZwQuerySystemInformation(

SystemHandleInformation,

pBuffer, cbBuffer * sizeof * pBuffer, NULL);

if (Status == STATUS_INFO_LENGTH_MISMATCH) {

delete [] pBuffer;

pBuffer = new ULONG[cbBuffer *= 2];

}

else if (!NT_SUCCESS(Status)) {

delete [] pBuffer;

return NULL;

}

} while (Status == STATUS_INFO_LENGTH_MISMATCH);

return pBuffer;

}

If a process opens a port, it must have created kernel objects of type \\Device\\Tcp or \\Device\\Udp. By opening these objects ourselves and saving their handles, we can search the handle list for entries matching our saved handles, then retrieve the corresponding kernel object addresses. Code:

DWORD TcpHandle;

DWORD UdpHandle;

DWORD GetTcpUdpObject(PULONG pBuffer, HANDLE pHandle, DWORD ProcessId)

{

DWORD objTYPE1, objTYPE2, HandleObject;

PSYSTEM_HANDLE_INFORMATION pProcesses = (PSYSTEM_HANDLE_INFORMATION)(pBuffer + 1);

for (i = 0; i < *pBuffer; i++) {

if ((pProcesses[i].ProcessID) == ProcessId) {

objTYPE1 = (DWORD)hDeviceTcpUdp;

objTYPE2 = (DWORD)pProcesses[i].Handle;

if (objTYPE1 == objTYPE2) {

HandleObject = (DWORD)pProcesses[i].Object;

return HandleObject;

}

}

}

return 0;

}

This kernel object address is a linear (virtual) address. We must convert it to a physical address and extract relevant data. In fport, the conversion is done as follows:

(See WebCrazy's article for details.)

void * NewmapPhy;

void GetPTE(DWORD objAddress)

{

DWORD physmemBuff;

DWORD newAddress1, newAddress2, newAddress3, newAddress4;

DWORD * newAddress;

physmemBuff = (DWORD)pMapPhysicalMemory;

newAddress1 = physmemBuff + (objAddress >> 0x16) * 4;

newAddress = (DWORD *)newAddress1;

newAddress1 = *newAddress;

newAddress2 = objAddress & 0x3FF000;

newAddress3 = newAddress1 & 0x0FFFFF000;

newAddress4 = newAddress2 + newAddress3;

NewmapPhy = MapViewOfFile(ghPhysicalMemory, FILE_MAP_READ, 0, newAddress4, 0x1000);

}

Next, we retrieve the physical page corresponding to the linear address and access the page containing the kernel object data. The PTE structure is:

typedef struct {

ULONG Present;

ULONG WriteTable;

ULONG User;

ULONG WriteThru;

ULONG NoCache;

ULONG Accessed;

ULONG Dirty;

ULONG PageSize;

ULONG Global;

ULONG Available;

ULONG Pfn;

} PTE, *PPTE;

(Note: This structure may not be fully accurate, but it suffices for our purposes.)

Code:

ULONG CurrWriteTable;

ULONG CurrNoCache;

void GetMustPar(DWORD objAddress)

{

DWORD CurrAddress = objAddress & 0xFFF;

PPTE pte = (PPTE)((DWORD)NewmapPhy + CurrAddress);

CurrWriteTable = pte->WriteTable;

CurrNoCache = pte->NoCache;

}

Now we have all necessary data. The next step is to iterate through all processes and their handles. (Note: Not all handles—on Windows NT, \\Device\\Tcp and \\Device\\Udp have object type 0x16; on Windows 2000, it's 0x1A.) For each such handle, we use the above method to retrieve its PTE and WriteTable value. If it matches that of \\Device\\Tcp or \\Device\\Udp, the handle likely represents an open port. We then confirm it. Confirmation code:

typedef struct _TDI_CONNECTION_INFO {

ULONG State;

ULONG Event;

ULONG TransmittedTsdus;

ULONG ReceivedTsdus;

ULONG TransmissionErrors;

ULONG ReceiveErrors;

LARGE_INTEGER Throughput;

LARGE_INTEGER Delay;

ULONG SendBufferSize;

ULONG ReceiveBufferSize;

BOOLEAN Unreliable;

} TDI_CONNECTION_INFO, *PTDI_CONNECTION_INFO;

typedef struct _TDI_CONNECTION_INFORMATION {

LONG UserDataLength;

PVOID UserData;

LONG OptionsLength;

PVOID Options;

LONG RemoteAddressLength;

PVOID RemoteAddress;

} TDI_CONNECTION_INFORMATION, *PTDI_CONNECTION_INFORMATION;

(See tdi.h)

void GetOpenPort(DWORD dwProcessesID, USHORT Handle, int NoCache)

// dwProcessesID: Process ID

// Handle: Handle of type \Device\Tcp or \Device\Udp

// NoCache: Value from PTE structure

{

HANDLE hProc, DupHandle = NULL;

HANDLE hEven = NULL;

OVERLAPPED overlap;

u_short openport;

int i = 0;

char procName[256] = {0};

int portflag = 0;

overlap.Internal = 0;

overlap.InternalHigh = 0;

overlap.Offset = 0;

overlap.OffsetHigh = 0;

hEven = CreateEvent(0, 1, 0, 0);

overlap.hEvent = hEven;

hProc = OpenProcess(PROCESS_DUP_HANDLE, 0, dwProcessesID);

if (hProc) {

DuplicateHandle(hProc, (HANDLE)Handle, GetCurrentProcess(), &DupHandle, 0, FALSE, 2);

CloseHandle(hProc);

if (DupHandle) {

TDI_CONNECTION_INFO TdiConnInfo = {0};

TDI_CONNECTION_INFORMATION TdiConnInformation = {0};

DWORD dwRetu = 0;

if (NoCache == 0x2) {

TdiConnInformation.RemoteAddressLength = 4;

if (DeviceIoControl(DupHandle, 0x210012,

&TdiConnInformation, sizeof(TdiConnInformation),

&TdiConnInfo, sizeof(TdiConnInfo), 0, &overlap)) {

openport = ntohs((u_short)TdiConnInfo.ReceivedTsdus);

procName = GetProcName(dwProcessesID);

printf("PID = %4d ProcessName = %15s PORT = %4d\n", dwProcessesID, procName, openport);

}

}

if (NoCache == 0x1) {

TdiConnInformation.RemoteAddressLength = 3;

if (DeviceIoControl(DupHandle, 0x210012,

&TdiConnInformation, sizeof(TdiConnInformation),

&TdiConnInfo, sizeof(TdiConnInfo), 0, &overlap)) {

openport = ntohs((u_short)TdiConnInfo.ReceivedTsdus);

procName = GetProcName(dwProcessesID);

printf("PID = %4d ProcessName = %15s PORT = %4d\n", dwProcessesID, procName, openport);

}

}

}

}

CloseHandle(hEven);

CloseHandle(DupHandle);

}

The above is my analysis and implementation code for fport.exe. The demo program is available for download from whitecell.org. If you find any issues, please let me know. ^_^

References:

fport.exe
Gary Nebbett, Windows NT/2000 Native API Reference
WebCrazy, On Windows NT/2000 Paging Mechanism
NTDDK

About Us:

WSS (Whitecell Security Systems) is a non-profit, grassroots technical organization dedicated to research in system security. We uphold traditional hacker ethics and strive for technical excellence.

WSS Homepage: http://www.whitecell.org/

WSS Forum: http://www.whitecell.org/forum/

Comprehensive Guide to Linux Processes, Signals, IPC, and gprof Usage (Reprinted)

Leo Liu — Sun, 17 May 2026 06:56:14 +0000

Originally published on the Sienovo Engineering Blog. Sienovo is the overseas brand of 深圳信迈 (Shenzhen Xinmai), building edge AI computing solutions for industrial video analytics.

Comprehensive Guide to Linux Processes, Signals, IPC, and gprof Usage (Reprinted)

Source: ChinaUnix Blog　Date: 2008.03.14 15:48　(Comments: ) Comment here Thanks to 007xiong, Hoyt, and others. This article is excellent. Concerned that the original link might disappear someday, I'm making a backup here. Original link:

http://www.mwjx.com/aboutfish/private/book/linux_c.txt

Introduction to C Programming under Linux

Compiled and written by: 007xiong

Original: Hoyt et al.

(1) Table of Contents

1) Introduction to Linux Program Design -- Basics

2) Introduction to Linux Program Design -- Processes

3) Introduction to Linux Program Design -- File Operations

4) Introduction to Linux Program Design -- Time Concepts

5) Introduction to Linux Program Design -- Signal Handling

6) Introduction to Linux Program Design -- Message Management

7) Introduction to Linux Program Design -- Thread Operations

8) Introduction to Linux Program Design -- Network Programming

9) Introduction to C Development Tools under Linux

(2) Detailed Content

1) Introduction to Linux Program Design -- Basics

Fundamentals of C Programming under Linux

Preface:

This article introduces the foundational knowledge required for C programming under Linux. In this article, we will learn:

Source code compilation

Writing Makefiles

Linking program libraries

Program debugging

Header files and system help

----------------------------------------------------------------------------

----

Source Code Compilation Under Linux, to compile a C source program, we use the GNU gcc compiler. Let's illustrate how to use gcc with an example. Suppose we have the following very simple source program (hello.c): int main(int argc,char **argv) { printf("Hello Linux\n"); } To compile this program, simply execute the following command: gcc -o hello hello.c The gcc compiler will generate an executable file named hello. Running ./hello will display the program's output. In the command line, gcc indicates we are using gcc to compile our source code, the -o option specifies that we want the output executable file to be named hello, and hello.c is our source file. The gcc compiler has many options, but generally, knowing just a few is sufficient. We already know the -o option, which specifies the desired name of the output executable file. The -c option means we only want the compiler to output object code without generating an executable file. The -g option tells the compiler to include debugging information for later program debugging. Knowing these three options enables you to compile simple source programs you write. If you want to know more options, refer to gcc's help documentation, which provides detailed explanations of other options.
Writing Makefiles Suppose we have the following program with the source code below: /* main.c */ #include "mytool1.h" #include "mytool2.h" int main(int argc,char **argv) { mytool1_print("hello"); mytool2_print("hello"); } /* mytool1.h */ #ifndef _MYTOOL_1_H #define _MYTOOL_1_H void mytool1_print(char *print_str); #endif /* mytool1.c */ #include "mytool1.h" void mytool1_print(char *print_str) { printf("This is mytool1 print %s\n",print_str); } /* mytool2.h */ #ifndef _MYTOOL_2_H #define _MYTOOL_2_H void mytool2_print(char *print_str); #endif /* mytool2.c */ #include "mytool2.h" void mytool2_print(char *print_str) { printf("This is mytool2 print %s\n",print_str); } Of course, since this program is very short, we could compile it like this: gcc -c main.c gcc -c mytool1.c gcc -c mytool2.c gcc -o main main.o mytool1.o mytool2.o This way, we can also generate the main program, and it's not too troublesome. But what if one day we modify one of the files (say, mytool1.c)? Would we have to re-enter the above commands? You might say this is easy to solve—just write a SHELL script to do it. Yes, for this program, that would work. But if we think more complexly, what if our program has hundreds of source files? Would we still have the compiler recompile each one individually? To solve this, clever programmers invented a great tool: make. By simply running make, we can resolve the above issue. Before running make, we must first write a very important file—Makefile. A possible Makefile for the above program is: # This is the Makefile for the program above main:main.o mytool1.o mytool2.o gcc -o main main.o mytool1.o mytool2.o main.o:main.c mytool1.h mytool2.h gcc -c main.c mytool1.o:mytool1.c mytool1.h gcc -c mytool1.c mytool2.o:mytool2.c mytool2.h gcc -c mytool2.c With this Makefile, no matter which source file we modify, running the make command will cause the compiler to compile only the files related to the modified file; it won't bother with the others. Next, let's learn how to write a Makefile. Lines starting with # in a Makefile are comments. The most important part of a Makefile is the dependency description. The general format is: target: components TAB rule The first line indicates the dependency relationship. The second line is the rule. For example, the second line of our Makefile above: main:main.o mytool1.o mytool2.o indicates that the target (main) depends on the components (main.o, mytool1.o, mytool2.o). When a dependent object is modified after the target, the command specified in the rule line must be executed. As stated in the third line of our Makefile, execute gcc -o main main.o mytool1.o mytool2.o. Note: The TAB in the rule line represents an actual TAB key. Makefiles have three very useful variables: $@, $<, and $^. $@ represents the target file, $< represents the first dependent file, and $^ represents all dependent files. For example, in the rule: main.o:main.c mytool1.h mytool2.h gcc -c main.c $@ is main.o, $< is main.c, and $^ is main.c mytool1.h mytool2.h.
Program Libraries After writing a program, we often need to link it with standard libraries. For example, the following program: #include #include int main(int argc,char **argv) { double value; printf("Value:%f\n",value); } This program is quite simple, but when we compile it with gcc -o temp temp.c, the following error appears: /tmp/cc33Kydu.o: In function main': /tmp/cc33Kydu.o(.text+0xe): undefined reference to \log' collect2: ld returned 1 exit status This error occurs because the compiler cannot find the specific implementation of log. Although we included the correct header file, we still need to link the appropriate library during compilation. Under Linux, to use mathematical functions, we must link with the math library, so we need to add the -lm option. Using gcc -o temp temp.c -lm will compile correctly. Someone might ask, why didn't we need to link a library when using printf earlier? The reason is that for some commonly used functions, the gcc compiler automatically links certain standard libraries, so we don't need to specify them ourselves. Sometimes, when compiling a program, we also need to specify the library path, in which case we use the compiler's -L option to specify the path. For example, if we have a library in /home/hoyt/mylib, we need to add -L/home/hoyt/mylib during compilation. For standard libraries, we don't need to specify the path as long as they are in the default library paths. The system's default library paths are /lib, /usr/lib, and /usr/local/lib. Libraries in these three paths can be used without specifying the path. Another issue: sometimes we use a function but don't know the library name. What should we do then? Unfortunately, I don't know the answer to this question either. I only have a crude method. First, I look in the standard library paths for a library related to the function I'm using. This is how I found the library file (libpthread.a) for thread functions. Of course, if I can't find it, there's only one clumsy method. For example, to find the library containing the sin function, I can only use the command nm -o /lib/*.so|grep sin>~/sin, then look in the ~/sin file. In the sin file, I find a line like libm-2.1.2.so:00009fa0 W sin, which tells me that sin is in the libm-2.1.2.so library, so I can use the -lm option (removing the leading lib and trailing version number, leaving just m, hence -lm). If you know a better method, please let me know immediately—I would be very grateful. Thank you!
Program Debugging Our programs are unlikely to succeed on the first try. Many unexpected errors may occur, requiring us to debug the program. The most commonly used debugging software is gdb. If you prefer a graphical interface, you can use xxgdb. Remember to include the -g option during compilation. For details on using gdb, refer to its help documentation. Since I haven't used this software much, I can't explain how to use it. However, I don't like using gdb. Tracing a program is quite tedious; I usually debug by outputting intermediate variable values within the program. Of course, you can choose your own method—there's no need to follow others. Now there are many IDE environments that include built-in debuggers. You can try a few to find one you like.
Header Files and System Help Sometimes we only remember the general form of a function but not the exact expression, or we don't remember which header file declared the function. In such cases, we can seek help from the system. For example, if we want to know the exact form of the fread function, we can simply run man fread, and the system will output a detailed explanation of the function and the header file where it is declared. If we want the description of the write function, running man write gives us the command description instead of the function description. To get the function description for write, we use man 2 write. The number 2 indicates that the write function is a system call. Another commonly used number is 3, indicating a C library function. Remember, man is always our best helper, no matter what. ------------------------------------------------------------------------ Well, that's enough for this chapter. With this knowledge, we can now embark on exciting C programming adventures under Linux. 2) Introduction to Linux Program Design -- Processes Process Creation under Linux Preface: This article introduces various concepts related to processes under Linux. We will learn: Concept of processes Process identity Process creation Daemon process creation ---------------------------------------------------------------------------- ----
Concept of Processes The Linux operating system is multi-user. At any given time, multiple users can issue various commands to the operating system. How does the operating system achieve a multi-user environment? Modern operating systems have concepts of programs and processes. What is a program, and what is a process? Loosely speaking, a program is a file containing executable code—a static file. A process, on the other hand, is an instance of a program that has started execution but has not yet finished. It is the concrete implementation of an executable file. A single program can have multiple processes, and each process can have multiple child processes, creating a hierarchy of parent and child processes. When a program is loaded into memory by the system, the system allocates certain resources (memory, devices, etc.) to it and performs a series of complex operations to turn the program into a process for system use. In the system, only processes exist, not programs. To distinguish between different processes, the system assigns each process an ID (like an ID card) for identification. To fully utilize resources, the system also categorizes processes into different states: new, running, blocked, ready, and completed. New means the process is being created; running means the process is executing; blocked means the process is waiting for an event; ready means the system is waiting for the CPU to execute; and completed means the process has finished and the system is reclaiming resources. For detailed explanations of the five process states, refer to "Operating Systems."
Process Identification As mentioned above, we know that processes have an ID. How do we obtain a process's ID? The system call getpid retrieves the process ID, and getppid retrieves the parent process ID (the process that created the calling process). #include pid_t getpid(void); pid_t getppid(void); Processes serve programs, and programs serve users. To identify the user associated with a process, the system links the process to a user, known as the process owner. Each user also has a user ID. The system call getuid retrieves the owner's ID. Since processes use resources, and Linux protects system resources, processes also have an effective user ID related to resource usage and permissions. The system call geteuid retrieves the effective user ID. Correspondingly, processes also have a group ID and an effective group ID. The system calls getgid and getegid retrieve the group ID and effective group ID, respectively. #include #include uid_t getuid(void); uid_t geteuid(void); gid_t getgid(void); gid_t getegid(void); Sometimes we are interested in other user information (such as login name), in which case we can call getpwuid to retrieve it. struct passwd { char *pw_name; /* Login name */ char *pw_passwd; /* Login password */ uid_t pw_uid; /* User ID */ gid_t pw_gid; /* Group ID */ char *pw_gecos; /* Real name */ char *pw_dir; /* User directory */ char *pw_shell; /* User shell */ }; #include #include struct passwd *getpwuid(uid_t uid); Next, let's practice using the functions we've learned with an example: #include #include #include #include int main(int argc,char **argv) { pid_t my_pid,parent_pid; uid_t my_uid,my_euid; gid_t my_gid,my_egid; struct passwd *my_info; my_pid=getpid(); parent_pid=getppid(); my_uid=getuid(); my_euid=geteuid(); my_gid=getgid(); my_egid=getegid(); my_info=getpwuid(my_uid); printf("Process ID:%ld\n",my_pid); printf("Parent ID:%ld\n",parent_pid); printf("User ID:%ld\n",my_uid); printf("Effective User ID:%ld\n",my_euid); printf("Group ID:%ld\n",my_gid); printf("Effective Group ID:%ld\n",my_egid); if(my_info) { printf("My Login Name:%s\n" ,my_info->pw_name); printf("My Password :%s\n" ,my_info->pw_passwd); printf("My User ID :%ld\n",my_info->pw_uid); printf("My Group ID :%ld\n",my_info->pw_gid); printf("My Real Name:%s\n" ,my_info->pw_gecos); printf("My Home Dir :%s\n", my_info->pw_dir); printf("My Work Shell:%s\n", my_info->pw_shell); } }
Process Creation Creating a process is simple. We just need to call the fork function. #include #include pid_t fork(); When a process calls fork, the system creates a child process. The only differences between the child and parent processes are their process IDs and parent process IDs; everything else is identical, as if the parent process cloned itself. Of course, creating two identical processes is meaningless. To distinguish between parent and child processes, we must track the return value of fork. When fork fails (due to insufficient memory or reaching the maximum number of processes for the user), fork returns -1. Otherwise, the return value of fork is crucial. For the parent process, fork returns the child process's ID, while for the child process, fork returns 0. We use this return value to distinguish between parent and child processes. Why does the parent process create a child process? As mentioned earlier, Linux is a multi-user operating system, and at any given time, multiple users compete for system resources. Sometimes, a process creates a child process to compete for resources to complete tasks sooner. Once the child process is created, both parent and child processes continue execution from the point of fork, competing for system resources. Sometimes, we want the child process to continue executing while the parent process blocks until the child process completes. In this case, we can call the wait or waitpid system calls. #include #include pid_t wait(int *stat_loc); pid_t waitpid(pid_t pid,int *stat_loc,int options); The wait system call blocks the parent process until a child process ends or the parent receives a signal. If the parent has no child processes or all child processes have ended, wait returns immediately. On success (due to a child process ending), wait returns the child process's ID; otherwise, it returns -1 and sets the global variable errno. stat_loc is the exit status of the child process, set by the child process via exit, _exit, or return. To obtain this value, Linux defines several macros to test the return value: WIFEXITED: Checks if the child process exited normally. WEXITSTATUS: Retrieves the exit status of the child process (valid when WIFEXITED is true). WIFSIGNALED: Checks if the child process was terminated by an unhandled signal. WTERMSIG: Retrieves the signal number that terminated the child process (valid when WIFSIGNALED is true). waitpid waits for a specified child process to return. If pid is positive, it waits for the specified process (pid). If pid is 0, it waits for any process in the same group as the caller. If pid is -1, it behaves like the wait call. If pid is less than -1, it waits for any process whose group ID equals the absolute value of pid. stat_loc has the same meaning as in wait. The options parameter determines the parent process's state and can take two values: WNOHANG, which causes the parent to return immediately if no child exists; and WUNTRACED, which causes waitpid to return when the child process ends, but the exit status is not available. After a parent process creates a child process, the child process usually needs to execute a different program. To invoke a system program, we can use the exec family of system calls. The exec family includes five functions: #include int execl(const char *path,const char *arg,...); int execlp(const char *file,const char *arg,...); int execle(const char *path,const char *arg,...); int execv(const char *path,char *const argv[]); int execvp(const char *file,char *const argv[]); The exec family can execute a given program. For detailed explanations of the exec family, refer to the system manual (man exec). Let's study an example. Note: compile with -lm to link the math library. #include #include #include #include #include #include #include void main(void) { pid_t child; int status; printf("This will demonstrate how to get child status\n"); if((child=fork())==-1) { printf("Fork Error :%s\n",strerror(errno)); exit(1); } else if(child==0) { int i; printf("I am the child:%ld\n",getpid()); for(i=0;i<1000000;i++); printf("I exit normally.\n"); exit(0); } else { printf("I am the parent:%ld\n",getpid()); wait(&status); if(WIFEXITED(status)) { printf("Child exited with code:%d\n",WEXITSTATUS(status)); } else if(WIFSIGNALED(status)) { printf("Child terminated by signal:%d\n",WTERMSIG(status)); } } }
Daemon Process Creation Daemon processes are background processes that run independently of terminals. They are often used for system services. To create a daemon process, follow these steps:
Call fork to create a child process, then exit the parent process. This ensures the child is not a process group leader.
Call setsid to create a new session, making the child process a session leader and detaching it from the controlling terminal.
Change the working directory to the root directory to prevent the current directory from being locked.
Close unnecessary file descriptors and redirect standard input, output, and error to /dev/null. Here is an example of a simple daemon process: #include #include #include #include #include #include #include #include /* Default user mailbox path in Linux is /var/spool/mail/username */ #define MAIL "/var/spool/mail/hoyt" /* Sleep for 10 seconds */ #define SLEEP_TIME 10 main(void) { pid_t child; if((child=fork())==-1) { printf("Fork Error:%s\n",strerror(errno)); exit(1); } else if(child>0) while(1); if(kill(getppid(),SIGTERM)==-1) { printf("Kill Parent Error:%s\n",strerror(errno)); exit(1); } { int mailfd; while(1) { if((mailfd=open(MAIL,O_RDONLY))!=-1) { fprintf(stderr,"%s","\007"); close(mailfd); } sleep(SLEEP_TIME); } } } You can create your mailbox file in the default path and test this program. Of course, this program has many areas for improvement. We will enhance this small program later. Before I show my improvements, try improving it yourself—perhaps allowing the user to specify the mailbox path and sleep time. Believe in yourself—you can do it. Go ahead, brave explorer. Well, that's enough for this section on processes. Processes are a very important concept, and many programs use child processes. Creating a child process is a fundamental requirement for every programmer! 3) Introduction to Linux Program Design -- File Operations File Operations under Linux Preface: In this section, we will discuss the various functions for file operations under Linux. File creation and read/write File attributes Directory operations Pipe files ---------------------------------------------------------------------------- ----
File Creation and Read/Write I assume you already know the standard-level file operation functions (fopen, fread, fwrite, etc.). If not, don't worry. The system-level file operations we discuss here actually serve the standard-level operations. When we need to open a file for read/write operations, we can use the system call open. After use, we call close to close the file. #include #include #include #include int open(const char *pathname,int flags); int open(const char *pathname,int flags,mode_t mode); int close(int fd); The open function has two forms. pathname is the filename (including path, default is current directory). flags can take one of the following values or a combination: O_RDONLY: Open for reading only. O_WRONLY: Open for writing only. O_RDWR: Open for both reading and writing. O_APPEND: Open in append mode. O_CREAT: Create a file. O_EXCL: If O_CREAT is used and the file exists, an error occurs. O_NOBLOCK: Open in non-blocking mode. O_TRUNC: If the file exists, truncate its contents. The first three flags can only be used singly. If O_CREAT is used, we must use the second form of open and specify the mode flag to indicate file access permissions. mode can be a combination of the following: ----------------------------------------------------------------- S_IRUSR User can read S_IWUSR User can write S_IXUSR User can execute S_IRWXU User can read, write, execute ----------------------------------------------------------------- S_IRGRP Group can read S_IWGRP Group can write S_IXGRP Group can execute S_IRWXG Group can read, write, execute ----------------------------------------------------------------- S_IROTH Others can read S_IWOTH Others can write S_IXOTH Others can execute S_IRWXO Others can read, write, execute ----------------------------------------------------------------- S_ISUID Set user ID S_ISGID Set group ID ----------------------------------------------------------------- We can also use numbers to represent the permission bits. Linux uses five digits to represent file permissions: 00000. The first digit sets the user ID, the second sets the group ID, the third represents user permissions, the fourth group permissions, and the fifth others' permissions. Each digit can be 1 (execute), 2 (write), 4 (read), 0 (none), or a sum of these values. For example, to create a file with user read/write/execute, group no permissions, others read/execute, and set user ID, we can use the mode --1 (set user ID) 0 (group not set) 7 (1+2+4) 0 (no permissions, use default) 5 (1+4), i.e., 10705: open("temp",O_CREAT,10705); If the file opens successfully, open returns a file descriptor. All subsequent file operations use this file descriptor. After operations are complete, close the file by calling close, where fd is the file descriptor to close. After opening a file, we perform read/write operations using the read and write functions. #include ssize_t read(int fd, void *buffer,size_t count); ssize_t write(int fd, const void *buffer,size_t count); fd is the file descriptor for read/write operations, buffer is the memory address for writing to or reading from the file, and count is the number of bytes to read/write. For regular files, read reads count bytes from the specified file (fd) into the buffer (remember to provide a sufficiently large buffer) and returns count. If read reaches the end of the file or is interrupted by a signal, the return value is less than count. If interrupted by a signal with no data returned, read returns -1 and sets errno to EINTR. When the program reaches the end of the file, read returns 0. write writes count bytes from buffer to file fd, returning the actual number of bytes written on success. Next, let's study an example that copies a file. #include #include #include #include #include #include #include #include #define BUFFER_SIZE 1024 int main(int argc,char **argv) { int from_fd,to_fd; int bytes_read,bytes_write; char buffer[BUFFER_SIZE]; char *ptr; if(argc!=3) { fprintf(stderr,"Usage:%s fromfile tofile\n\a",argv[0]); exit(1); } /* Open source file */ if((from_fd=open(argv[1],O_RDONLY))==-1) { fprintf(stderr,"Open %s Error:%s\n",argv[1],strerror(errno)); exit(1); } /* Create destination file */ if((to_fd=open(argv[2],O_WRONLY|O_CREAT,S_IRUSR|S_IWUSR))==-1) { fprintf(stderr,"Open %s Error:%s\n",argv[2],strerror(errno)); exit(1); } /* The following code is a classic file copy */ while(bytes_read=read(from_fd,buffer,BUFFER_SIZE)) { /* A fatal error occurred */ if((bytes_read==-1)&&(errno!=EINTR)) break; else if(bytes_read>0) { ptr=buffer; while(bytes_write=write(to_fd,ptr,bytes_read)) { /* A fatal error occurred */ if((bytes_write==-1)&&(errno!=EINTR))break; /* All read bytes written */ else if(bytes_write==bytes_read) break; /* Only part written, continue */ else if(bytes_write>0) { ptr+=bytes_write; bytes_read-=bytes_write; } } /* Fatal error during write */ if(bytes_write==-1)break; } } close(from_fd); close(to_fd); exit(0); }
File Attributes Files have various attributes, including creation time, size, etc., in addition to permissions. Sometimes, we need to check if a file can be accessed in a certain way (read, write, etc.). In this case, we can use the access function. #include int access(const char *pathname,int mode); pathname is the filename, and mode is the attribute we want to check. It can take the following values or combinations: R_OK (readable), W_OK (writable), X_OK (executable), F_OK (file exists). The function returns 0 on success, or -1 if any condition fails. To obtain other file attributes, we can use the stat or fstat functions. #include #include int stat(const char *file_name,struct stat *buf); int fstat(int filedes,struct stat *buf); struct stat { dev_t st_dev; /* Device */ ino_t st_ino; /* Inode */ mode_t st_mode; /* Mode */ nlink_t st_nlink; /* Hard links */ uid_t st_uid; /* User ID */ gid_t st_gid; /* Group ID */ dev_t st_rdev; /* Device type */ off_t st_off; /* File size in bytes */ unsigned long st_blksize; /* Block size */ unsigned long st_blocks; /* Number of blocks */ time_t st_atime; /* Last access time */ time_t st_mtime; /* Last modification time */ time_t st_ctime; /* Last change time (metadata) */ }; stat is used for files not opened, while fstat is for opened files. The most commonly used attribute is st_mode. We can use the following macros to determine if a given file is a regular file, directory, link, etc.: S_ISLNK(st_mode): Is it a symbolic link? S_ISREG: Is it a regular file? S_ISDIR: Is it a directory? S_ISCHR: Is it a character device? S_ISBLK: Is it a block device? S_ISFIFO: Is it a FIFO file? S_ISSOCK: Is it a socket file? We will explain how to use these macros below.
Directory Operations When writing programs, we sometimes need to get the current working directory. The C library function getcwd solves this problem. #include char *getcwd(char *buffer,size_t size); We provide a buffer of size size, and getcwd copies the current path into the buffer. If the buffer is too small, the function returns -1 and an error code. Linux provides many directory operation functions. We'll learn a few simple and commonly used ones. #include #include #include #include #include int mkdir(const char *path,mode_t mode); DIR *opendir(const char *path); struct dirent *readdir(DIR *dir); void rewinddir(DIR *dir); off_t telldir(DIR *dir); void seekdir(DIR *dir,off_t off); int closedir(DIR *dir); struct dirent { long d_ino; off_t d_off; unsigned short d_reclen; char d_name[NAME_MAX+1]; /* File name */ mkdir is straightforward—it creates a directory. opendir opens a directory for reading. readdir reads an open directory. rewinddir is used to re-read a directory, similar to the rewind function. closedir closes a directory. telldir and seekdir are similar to ftell and fseek. Next, we'll develop a small program that takes one parameter. If the parameter is a filename, it outputs the file's size and last modification time. If it's a directory, it outputs the size and modification time of all files in that directory. #include #include #include #include #include #include #include #include #include static int get_file_size_time(const char *filename) { struct stat statbuf; if(stat(filename,&statbuf)==-1) { printf("Get stat on %s Error:%s\n", filename,strerror(errno)); return(-1); } if(S_ISDIR(statbuf.st_mode))return(1); if(S_ISREG(statbuf.st_mode)) printf("%s size:%ld bytes\tmodified at %s", filename,statbuf.st_size,ctime(&statbuf.st_mtime)); return(0); } int main(int argc,char **argv) { DIR *dirp; struct dirent *direntp; int stats; if(argc!=2) { printf("Usage:%s filename\n\a",argv[0]); exit(1); } if(((stats=get_file_size_time(argv[1]))==0)||(stats==-1))exit(1); if((dirp=opendir(argv[1]))==NULL) { printf("Open Directory %s Error:%s\n", argv[1],strerror(errno)); exit(1); } while((direntp=readdir(dirp))!=NULL) if(get_file_size_time(direntp->d_name)==-1)break; closedir(dirp); exit(0); }
Pipe Files Pipes are a form of inter-process communication (IPC) that allow data to be passed between processes. A pipe has two ends: one for ---