Why gdb Shows ?? Instead of Function Names: 4 Causes and Fixes

Akihiro Oguri — Mon, 12 Oct 2026 13:51:31 +0000

Introduction

Sometimes you look at a backtrace in gdb and find ?? where the function names should be.

#0  0x0000aaaaaaaa0894 in ?? ()
#1  0x0000aaaaaaaa0994 in ?? ()

?? means that gdb could not find a function name for that address. This article covers four reasons you get ??:

The application has been stripped
The function is inside a library and is not public
The location of the libraries was not given with set sysroot
When a core dump is opened on the host, gdb cannot match it with the application

The test environment is ARM64 Linux (BusyBox) running on QEMU, with an Ubuntu 24.04 host.

Reason 1: The Application Has Been `strip`ped

Applications shipped in products are sometimes run through strip to make the file smaller. strip removes the following two things:

What is removed	Contents	What gdb uses it for
Symbols	The mapping between function and variable names and addresses	Turning addresses into function names
Debug information	Line numbers, the types and locations of variables, and so on (added with `-g`)	Showing line numbers and the values of variables

The symbols that function names come from are gone, so every function in the application becomes ??.

(No debugging symbols found in ./sensor-stripped)
...
Program received signal SIGSEGV, Segmentation fault.
0x0000aaaaaaaa0894 in ?? ()
(gdb) bt
#0  0x0000aaaaaaaa0894 in ?? ()
#1  0x0000aaaaaaaa0994 in ?? ()
#2  0x0000fffff7e184c4 in ?? () from /lib/libc.so.6
#3  0x0000fffff7e18598 in __libc_start_main () from /lib/libc.so.6
#4  0x0000aaaaaaaa0770 in ?? ()

The file command tells you whether a file has been stripped. If it says stripped at the end, it has.

$ file sensor-stripped
sensor-stripped: ELF 64-bit LSB pie executable, ARM aarch64, ... stripped

Fix: Keep the Unstripped Application or a Debug Information File

Even if the application on the target stays stripped, if you keep the unstripped application from the same build on the host, gdb on the host can show function names and line numbers when you connect through gdbserver.

(gdb) bt
#0  print_average () at sensor.c:33
#1  0x0000aaaaaaaa0994 in main () at sensor.c:51

Instead of keeping the whole unstripped application, you can also put just the debug information in a separate file.

objcopy --only-keep-debug sensor sensor.debug
objcopy --add-gnu-debuglink=sensor.debug sensor-stripped

The first line extracts only the debug information and symbols into sensor.debug. The second line writes a marker into the stripped application saying "the debug information is in sensor.debug." gdb follows this marker and loads sensor.debug (see "Debugging Information in Separate Files" in the gdb manual: https://sourceware.org/gdb/current/onlinedocs/gdb.html/Separate-Debug-Files.html).

With either method, what you keep must come from the same build. With something rebuilt later, the instructions and addresses may shift, and they cannot be matched correctly.

Note: Just Leaving Out `-g` Does Not Give You `??`

If you build without -g, the symbols remain, so function names still appear. What you lose are line numbers and variables. You get ?? only when strip removes the symbols as well.

Reason 2: The Function Is Inside a Library and Is Not Public

Sometimes the application itself is not stripped, but only the frames inside a library are ??.

(gdb) bt
#0  0x0000fffff7e92b1c in ?? () from /lib/libc.so.6
#1  0x0000aaaaaaaa0874 in reset_average () at sensor.c:27
#2  0x0000aaaaaaaa093c in main () at sensor.c:46

Line 27 in #1 is the line that calls memset(), so #0 is inside memset(). Even so, its name is ??.

The C library may keep only the names of the functions it exports. memset() actually works by running an implementation function chosen to suit the CPU, and that implementation function is not exported. So its name is shown as ??.

Fix: Look at the Calling Line

Even if you do not know the name of the function inside the library, bt tells you which line of your own code called it. In the example above, sensor.c:27 in #1 leads you to the memset() that caused the problem.

If you do not need to follow it into the library, the shortcut is to skip the ?? frames and look at the topmost frame in your own code.

Reason 3: The Location of the Libraries Was Not Given with `set sysroot`

An application runs using shared libraries such as the C library. gdb on the host reads the function names inside a library from the library's file.

When you open a core dump with gdb on the host alone, you need to tell gdb with set sysroot where on the host the target's library files are. If you do not, gdb cannot read the library files, and the frames inside the libraries become ??.

When you are connected through gdbserver, gdb receives the library files from the target through gdbserver and reads them, even without set sysroot.

Fix: Prepare a Directory Laid Out Like the Target and Specify It

gdb treats the directory specified with set sysroot as / on the target. For example, if you specify ~/sysroot, gdb looks for /lib/libc.so.6 on the target at ~/sysroot/lib/libc.so.6 on the host.

(gdb) set sysroot ~/sysroot
(gdb) file ./sensor
(gdb) core-file core

So inside the directory you specify, you need to place the libraries at the same paths as on the target. The gdb manual (Commands to Specify Files: https://sourceware.org/gdb/current/onlinedocs/gdb.html/Files.html) says:

If you use set sysroot to find executables and shared libraries, they need to be laid out in the same way that they are on the target, with e.g. a /bin, /lib and /usr/lib hierarchy under path.

Use the same library files as the ones running on the target. Specify set sysroot before loading the core dump.

Reason 4: When a Core Dump Is Opened on the Host, gdb Cannot Match It with the Application

Even after specifying set sysroot as in the fix for Reason 3, you may still get ??. Here is an example where only the libraries were placed there, not the application. Even though the application is given with file, neither function names nor line numbers appear.

(gdb) set sysroot ~/appdbg-study/initramfs
(gdb) file ./sensor
Reading symbols from ./sensor...
(gdb) core-file core
warning: Can't open file /mnt/ch05/sensor during file-backed mapping note processing
...
#0  0x0000aaaae5490894 in ?? ()
(gdb) bt
#0  0x0000aaaae5490894 in ?? ()
#1  0x0000ffff81c884c4 in ?? ()
#2  0x0000aaaae5490770 in ?? ()

Cause

Before gdb on the host applies function names, it checks that the application in the core dump and the application given with file are the same. What it uses for this check is the start of the application (the ELF headers), and gdb looks for it in this order:

Inside the core dump
In the application at the same path as on the target, inside the directory specified with set sysroot

If it is in neither place, function names become ??.

In this environment, it was in neither place.

The ELF headers were not in the core dump: whether they are included is decided by the 0x10 bit of /proc/<pid>/coredump_filter (bit 4 in man 5 core, "Dump ELF headers."). On the target, the value was 00000023, and this bit was not set
The application was not placed in the sysroot: the libraries were there, but the application (/mnt/ch05/sensor) was not

This default is decided by the kernel configuration CONFIG_CORE_DUMP_DEFAULT_ELF_HEADERS. In the kernel's Kconfig (fs/Kconfig.binfmt), it defaults to enabled, but the ARM64 defconfig disables it. In kernels distributed by board makers, it was also disabled in NXP's imx_v8_defconfig and in Jetson Linux's defconfig / tegra_defconfig.

Fix: Make One of the Two Available

Fix	What to do	When to do it
Place the application in the sysroot	Inside the directory specified with `set sysroot`, place the application at the same path as on the target too	After the crash, on the host
Put the ELF headers in the core dump	Set `coredump_filter` to `0x33`	Before the crash, on the target

If you set coredump_filter in the shell, it applies to the applications run from that shell.

echo 0x33 > /proc/self/coredump_filter
./sensor

With a core dump taken this way, function names and line numbers appeared even without placing the application in the sysroot.

#0  print_average () at sensor.c:33
33      printf("%s average: %d\n", avg.sensor->name, avg.sum / avg.num);
(gdb) bt
#0  print_average () at sensor.c:33
#1  0x0000aaaac4cd0994 in main () at sensor.c:51

You can change the default by adding coredump_filter=0x33 to the kernel boot options, or by enabling CONFIG_CORE_DUMP_DEFAULT_ELF_HEADERS and rebuilding the kernel.

Note that what goes into the core dump is only the ELF headers, not the whole files. With either fix, the shared libraries must be placed inside the sysroot at the same paths as on the target.

Summary

Reason	Where `??` appears	Fix
The application has been `strip`ped	Every function in the application	Keep the unstripped application from the same build, or a debug information file
A library function that is not public	Frames inside the library	Look at the calling line in your own code
The location of the libraries was not given with `set sysroot`	Frames inside the libraries	Prepare a directory laid out like the target and specify it with `set sysroot`
gdb cannot match the core dump opened on the host with the application	Every function in the application	Place the application in the sysroot, or set `coredump_filter` to `0x33`

When you see ??, first check which frames are ??; that gives you an idea of which reason applies. If everything, even inside the application, is ??, it is Reason 1 or 4. If only the frames inside libraries are, it is Reason 2 or 3.

Want to Learn More?

This article is based on my book, Hands-On Linux Application Debugging with QEMU.

In the book, with no real hardware, using just one PC, you debug applications for embedded Linux both on the target and from the host. You run applications with bugs put in on purpose and use tools to find the line that causes each one.

Stop an application in gdb, look at its variables, and get a backtrace. Find the line that overwrote a value with a watchpoint
Connect from the host with gdbserver and VS Code to debug
Open the core dump of a crashed application, both on the target and on the host
In a multithreaded application that has stopped, see which thread is waiting for what
Find memory errors that do not cause a crash with ASan and UBSan

amazon.com

DEV Community: Akihiro Oguri