I'm an expert-level C and C++ developer, with a specialty in memory management. I have experience writing memory-safe code with both the modern saf...
For further actions, you may consider blocking this person and/or reporting abuse
Hello Jason! I am encountering a peculiar issue. On one system after compiling I am getting no errors with pthreads in C++. On the second I am running across a Segfault
On the other I am recieving no errors. My wild guess is that the issue might stem from the systems having different kernels:
No Problems:
cat /proc/version
Linux version 4.15.0-29-generic (buildd@lgw01-amd64-057) (gcc version 7.3.0 (Ubuntu 7.3.0-16ubuntu3)) #31-Ubuntu SMP Tue Jul 17 15:39:52 UTC 2018
Problems(SegFault):
cat /proc/version
Linux version 5.4.0-42-generic (buildd@lgw01-amd64-038) (gcc version 9.3.0 (Ubuntu 9.3.0-10ubuntu2)) #46-Ubuntu SMP Fri Jul 10 00:24:02 UTC 2020
Please let me know if any other information is needed
Every time I see "No such file or directory," the first thing I check is the path. What is the current working directory from which you're running
a.out, and does the path../sysdeps/unix/sysv/linux/raise.cindeed exist relative to that working directory? (Remember that..means "parent directory".)My working path is the same as the pwd for where I am running
a.out. In this case it would be something on the lines of/home/red/recorder. I have no idea of where the path../sysdeps/unix/sysv/linux/raise.cis coming from, so I would say no it does not exist. I am assuming it is a segfault of some variant when I receive a response ofdouble free or corruption (out)andAborted (core dumped).I can provide a recent valgrind and or gdb bt if needed.
Yeah, that relative path is spooky.
To diagnose the double free or corrupted, you'd want to run your program through Valgrind. Meanwhile, you may need to use
gdbto step through your program (compiled with-g) to determine precisely when control leaves your code, onward to the abort.Alright the first block is a gdb output with a backtrace:
This is the valgrind dump (sorry for double comment; couldn't get the markdown to capture all of the code cleanly):
The most puzzling part is that I get this error on one system but not the other:
No Problems:
cat /proc/version
Linux version 4.15.0-29-generic (buildd@lgw01-amd64-057) (gcc version 7.3.0 (Ubuntu 7.3.0-16ubuntu3)) #31-Ubuntu SMP Tue Jul 17 15:39:52 UTC 2018
Problems(SegFault):
cat /proc/version
Linux version 5.4.0-42-generic (buildd@lgw01-amd64-038) (gcc version 9.3.0 (Ubuntu 9.3.0-10ubuntu2)) #46-Ubuntu SMP Fri Jul 10 00:24:02 UTC 2020
You must remember that undefined behavior is exactly that. It may appear to work, and then not work elsewhere. When it doesn't work, anything could happen, including "making demons fly out your nose". So, I'd start by going through that Valgrind output, bit by bit, and fixing each problem in your code it highlights. (The error and location of that error in your code is on the last line of each traceback block in Valgrind.)
Once your //own// code runs Valgrind pure, we can tackle any remaining weirdness.
After furious placements of
std::couteverywhere, I discovered that my function that my thread was calling was missing its return statement. Not sure how my first system was able to bypass that issue but it arised on my second system.So, it's resolved, then?
The issue yes. My curiosity, not so much lol. It shouldn't have compiled however that behavior I believe isactually allowed, it's grandfathered in from C. Since the function returns a variable and a return is not explicitly called then the first value in the stack frame is reinterpreted as the return type and returned instead
int foo(int bar) {
int blah= 8;
blah += 4;
int zayxxy = 0;
} //returns 12
So I believe that is the reason how my first system was able to execute without any problems. However, I do not fully understand how on the second system the function was never able to terminate the function but stay stuck in a while loop and result in a segfault.
What you described, not returning a value from a non-
voidfunction, is actually undefined behavior in C. Therefore, once again, it is legal for the compiler to make demons fly out your nose. Anything can happen. There is no rhyme or reason.Here's C99 on it — ISO/IEC 9899:1999, section 6.9.1 paragraph 12:
One system's compiler was able to figure it out anyway, and it worked, which is legal (because anything is). The other system's compiler was not, and it had a snit.
P.S. Thanks for asking! I learned something new today, namely that the above is undefined behavior.
Glad I could help! I have definitely learned alot from this experience as well!
OK, firstly, a small nitpick, you weren't getting a segfault (SIGSEGV) but a SIGABRT
As for failing to return a value from a non-void function, you should at the very least compile with -Wall, which would have caught that. e.g
vs
And of course we also get the second warning...
I always compile with at least '-Wall -Wextra'
Hello, I am learning robotics and using ros-kinetic with gazebo7. I am trying to launch my model in gazebo but got stuck on a "segmentation fault(core dump)" error at
0x00007fffc96ac0ed in ros::NodeHandle::destruct() ()
from /opt/ros/kinetic/lib/libroscpp.so
Kindly advice
There are only two ways to debug a segmentation fault, ordinarily:
1) If you have access to the source code for ros-kinetic, you would need to compile it yourself with the
-gflag (debug flag), and then try to use it the same way as before. Then, when the segfault occurs, you'll get a file and line number instead of the raw memory address (0x00007fffc96ac0ed), and that will tell you where in the code the segfault is (probably) happening from.2) To get more information, you can run the code (again, compiled with -g) through a dynamic memory analyser like Valgrind. That will not only give you the file and line number where the segfault is probably occurring, but also a hint about what's going on, and possibly a longer stack track.
Given the information from (1) or (2) (and a snippet of the offending source code), I could probably help you from there.
However, if ros-kinetic is not your project, you'll be best off filing a bug report on their issue tracker.
Thanks for the advice. I did compile ros-kinetic from source but now gdb wasn't launching, I don't know why. So i reinstalled ros-kinetic from apt and ran it, gdb was working. Well I did find the source file for the function pointing to segmentation fault :-
The backtrace went till 24 frames I could provide them too if the fault is not in this part of code.
If you could help me find the error it would really boost my learning.
The stack trace would be really helpful. Also, please be sure to precede your code example with three backticks (`) on the line above the example, and three on the line below.
The stack trace is here :-
Awesome, and the code you posted earlier, is that the context for
/opt/ros/kinetic/include/realtime_tools/realtime_publisher.h:84?Also, what is the rest of the Valgrind output? Any more details? Segfaults have many causes, so knowing which one was detected helps narrow down the problem.
The code I posted earlier is the context for frame 1&2. Actually this is the gdb output which I posted.
Well I just ran the same in valgrind and it gave:-
Speaking of any more details, I have Ubuntu xenial and trying to use my "launch file" to launch my robotic model in gazebo7(robotic simulation software) and this simulation software is giving segmentation fault on running my launch file. Since this launch file is readymade from Github I think probably there is no error in that launch file.
What do you think is causing the error based on my provided information?
o.O
Wow, I've never seen this one before. The segfault is occurring, but Valgrind doesn't seem to be catching it.
I'm curious how you're invoking Valgrind. Usually I'd just pass the executable right to it:
I invoked valgrind by specifying it as an option in the launch file itself and the same way i invoked gdb.
I am very stressed with this problem but i don't want to give up.
What do you suggest for this problem?
You know, I'd be really curious to know what would happen if you ran the launch file itself through Valgrind! If you look at the output from a moment ago, there's quite a lot that is occuring outside of Valgrind (all the lines not preceded with
==nnnnn==(wherennnnnis some number). The segfault at the end appears to be occuring outside of that context as well. That leads me to believe the segfault might actually be within the launch file.I just ran it through valgrind
Yikes. Could you delete that comment chain and put it in a Gist or bpaste.net or some such? It'll be easier to read.
In any case, that confirmed my suspicion; the launcher is the problem. it's not memory pure at all.
After ending the process manually I further got the output
So this was the whole output I got , sorry for uploading this in parts(character limitation).
I hope this gives something useful to track down the issue.
I apologise for making such a long comment chain.
I have now made a gist of running the launch file through valgrind in
gist.github.com/rishabh900/41fd6df...
And the above comment is the output after i terminated the process manually.
So what do you think of now?
Did you write the launcher script, or is that third-party? It's clearly written in Python, and the issue is definitely there. I just can't narrow in on the specific issue, because the memory issues are being thrown by the interpreter (e.g.
at 0x41964F: PyObject_Free (in /usr/bin/python2.7)). That indicates that something odd has been done within the Python code, but I won't be able to diagnose this further without really fully understanding the launcher's source code, and I'm afraid I don't have time to learn it.If this is third-party code, open an issue against the launcher project, and include the above output of Valgrind.
Great to see a fellow low-level programmer on here!
I worked on a game engine written in C and was having many issues related to wrongly using the
reallocfunction for dynamically allocated memory. What I did was forget to assign the reallocated memory's pointer to the return value of the function. It took me weeks before I found the underlying problem since only in some cases it would blow up. How would you go about debugging a situation like:Do you use some sort of special tools? Or just some coding standards to not let this happen?
Whenever I'm working with memory, I pair two different tools: Valgrind and Goldilocks (PawLIB).
Valgrind is a pretty ubiquitous tool on UNIX platforms which will show me all of the memory issues encountered while running, even if the undefined behavior doesn't cause any overt problems. My code isn't done until it's Valgrind-pure. However, Valgrind only monitors the execution, so...
Goldilocks is a testing framework I developed at MousePaw Media, as a part of PawLIB. You could technically use any testing framework, but the benefit to Goldilocks is that it bakes the tests into the final executable, instead of requiring an additional framework to run the tests. That way, you can start the normal executable, run each of the tests you wrote, and see which ones Valgrind complains about.
Mind you, this does require you to write a lot of comprehensive behavioral tests...but you really should be doing that anyway in production code. ;)
My approach for this specific problem is to use a compiler that warns about unused return value, such as gcc or clang. I know that stdlib.h on Linux and Mac OS X already decorates realloc() with warn_unused_result attribute.
stackoverflow.com/a/2889601
But just naively setting
p = realloc(p, ...)is also wrong, since if the allocation fails, p would be set toNULLbut the original object is still allocated. The original pointer is lost and now a memory leak. Use reallocf() which frees the original memory if it could not be resized.That's a really nice feature, didn't know about it.
But wouldn't that mean data loss in case the memory can't be resized? Wouldn't that become an unrecoverable error?
@liulk Ha, I completely forgot to mention Clang! It does indeed have the best warnings of any compiler I've used. I almost always compile with
-Wall -Wextra -Wpedantic -Werror; that last one (as you know, although the reader might not) causes the build to fail on any warnings.I also use
cppcheckas part of my autoreview workflow, and resolve all linter warnings before committing to the production branch.@codevault You're right, reallocf() would just free the memory and cause data loss, so it would serve a different use case than realloc(). The more general solution would be to always use this pattern, which is more verbose:
I just find that in most of my use cases, I would end up freeing p in the error handling, so I would just use reallocf() which results in less verbose code.
I see, that makes sense. I can see myself freeing the memory most of the time when reallocation fails.
Good to note. Thanks!
I should add, I use another tool from PawLIB called IOChannel - basically, a
std::coutwrapper - that allows me to cleanly print the address and raw memory from literally any pointer, without having to use a debugger. This can make debugging some problems infinitely easier, especially when you're contending with a Heisenbug that goes away if compiled with-g, but appears when compiled with-O2.Thanks for the response!
Unfortunately, I didn't find a version of Valgrind for Windows. I tried DrMemory but, after lots of struggle, it didn't give me any helpful information and dropped the ball. Do you have experience with low-level on Windows or just work exclusively on Linux since it is more convenient?
I rarely use Windows for development, as its development toolchain is almost invariably miles behind its UNIX-based counterparts.
If you're on Windows 10, I strongly recommend setting up the Windows Subsystem for Linux [WSL]. That will give you access to the Linux development environment for compiling and testing. Then, use the LLVM Clang compiler on both the WSL and the Visual Studio environments. That way, once you know it compiles and runs Valgrind-pure on WSL, you can trust that it will work on VS Clang.
RE my previous post, (which I do not see): I figured out the problem. As I suspected, when I traced the spaghetti logic, I was doing something with undefined behavior. Fixed it, and there is no problem on either instance now. Thanks for all your contribution!
Are rust and golang going to take over C and C++? In terms of desktop software/web development. Also how would you, as a C++ expert, rate these languages? Do they have potential?
I strongly believe that (virtually) all languages have their place. FORTRAN and COBOL have firmly established places in the world, and are almost certain never to lose them on account of their reliability and precedence.
C and C++ likewise have this precedence, making up a sizable chunk of our source code. It's the old "if it ain't broke, don't fix it" concept; I doubt the entire collection of software that makes up a standard Linux-based operating system will ever be rewritten from C to Rust, because most of what already exists works quite well.
That said, I think Rust and golang have a lot of potential as languages, especially Rust.
(In my personal opinion, golang is a rather hipster language, but that's based in my feelings towards it, not in anything practical; so take that with a grain of salt.)
Rust looks especially interesting in the area of error handling. I'll admit, I haven't had the time to learn it very well yet, but it's DEFINITELY high on my list!
In other words, Rust and golang will probably find established places in the programming world, but they won't be displacing C, C++, or any other established language. Every tool has its place, and a quarter inch drill bit doesn't replace a 5mm drill bit.
Rust would take over everything, but the leftist collectivist community threw a baton roue of so-called "golang" at it, and now we will all perish and return to a dark age of literal witch hunts. (Because we are all out of Moore's law)
RIP information technology and human civilization in general
LOL
Nihilist
Hello Jason!! I am having some problem of memory corruption sometimes. Could you please help me out with some possible cause. Image is attached.
This issue is not always happening but sometimes. I have some memory crunch in my arm board. Is this the cause of this undefined behavior?
dev-to-uploads.s3.amazonaws.com/up...
Yes, this is undefined behavior. The line marked
#3indicates a double-free is occuring somewhere in your code, wherein an already-freed pointer is being freed again. The behavior of a double-free is undefined.Unfortunately, I don't have enough context here (especially in an image) to debug this for you. It looks like you're already using
valgrind. Try compiling your own code with debug flags (-g), and then running it through Valgrind and repeating whatever action triggers this error. The resulting stack trace should point you to the line in your code where the double-free is originating.Thank you very much for your reply.
I am unable to use valgrind on target due to very limited system memory on my embedded device. Could you please suggest any alternate approach which use low resource?
I have compiled my application with -ggdb3.
I have encounter this problem only with wide character text (std::wstring) and during the object destruction (out of scope) this corruption is happening.
Images
dev-to-uploads.s3.amazonaws.com/up...
dev-to-uploads.s3.amazonaws.com/up...
dev-to-uploads.s3.amazonaws.com/up...
dev-to-uploads.s3.amazonaws.com/up...
dev-to-uploads.s3.amazonaws.com/up...
dev-to-uploads.s3.amazonaws.com/up...
dev-to-uploads.s3.amazonaws.com/up...
Memory dump of variable from frame #6 of stack trace
dev-to-uploads.s3.amazonaws.com/up...
Could you please provide any hint?
If you have memory bugs in your code, it should also work if you compile and test it on your regular machine, even though the architecture is different. Have you tried that?
Thanks for your advice. I haven't tried to run this on regular pc. This is pretty big task but let me try this. Could you please provide any opinion to use ASAN compiled binary on target?
That might work. I haven't worked with ARM boards or microcircuits before, so I don't have much insight there.
Hi Jason,
I had a problem when mixing C and C++ code. Everything compiles just fine but when I run it I get a segfault. This happens when the code goes from C++ to C
I implemented it as follows:
include "Signature.hpp"
include
include
extern "C"{
#include "sign.h"
#include "sign.c"
#include "params.h"
#include "aes256ctr.h"
#include "aes256ctr.c"
#include "ntt.h"
#include "ntt.c"
#include "packing.h"
#include "packing.c"
#include "poly.h"
#include "poly.c"
#include "polyvec.h"
#include "polyvec.c"
#include "reduce.h"
#include "reduce.c"
#include "randombytes.h"
#include "randombytes.c"
#include "fips202.h"
#include "fips202.c"
#include "rounding.h"
#include "rounding.c"
#include "symmetric.h"
#include "symmetric-aes.c"
}
using namespace std;
// function to generate a keypair, mainly uses the c function from the Dilithium PQ scheme
void Signature::generateKeyPair(){
uint8_t* pk = getPublicKeyAddress();
uint8_t* sk = getSecretKeyAddress();
PQCLEAN_DILITHIUM5AES_CLEAN_crypto_sign_keypair(pk, sk);
cout << "I succeeded" << endl;
}
I really thought I implemented this right and I do not know what I am doing wrong. Could you help me?
For a start, you might want to wrap your code in your post above with three backticks on the line above, and three on the line below. Otherwise, it's hard to read the code. ;)
Dear Jason,
I am so sorry, this is my first time on the platform. I have used valgrind and this gives me that the segfault is caused by an invalid write of size 8. Here is my code (main function):
and this is the signature class from which the c code is called:
The PQCLEAN_DILITHIUM5AES_CLEAN_crypto_sign is a c function that was built for the NIST competition for post quantum signature schemes.
Do you know what I am doing wrong here?
More importantly, what file name and line number does it say that invalid write occurs on? (Consider posted the full output of Valgrind's error message, with traceback.)
Here is the valgrind error message. The problem happens inside another file than the two above this one although the problem can't be there because, as said above, this is code from an almost NIST standard in post quantum and has been reviewed in correctness. For completeness, here is the code:
Thank you btw for helping me!
The problem isn't always at the end of the stack trace, especially where memory management is involved. I don't think the library is at fault.
Actually, I think the problem is somewhere in
Signature.cpp. Can you post the function in that file that includes line 73?Okay, The Signature.cpp function in which this happens is a rather short one:
PQCLEAN_DILITHIUM5AES_CLEAN_crypto_sign is the c function from the library
So I think the problem might be even further back, in main.cpp, but I cannot figure out what I am doing wrong.
The main.cpp is written as follows:
I think I found it. The problem is in main.cpp:
There are two problems here.
sizeof(*sm)gives you the length of the value pointed to bysm. This will not give you the length of the string, but rather the size of the first item in the string/array, being a character. In other words, the value here is1, regardless of how long the string starting at pointersmactually is. I'll come back to this.Even if
smLengthwere correct,(size_t*) smLength;is not going to give you the address tosmLengthfor storing in the pointer. Instead, this is casting, or reinterpreting, the value, the integer1in this case, as a pointer. But what's at pointer0x00000001? Who knows. This is known as a wild pointer.If you look throughout your code, you'll see these same mistakes occur other times, such as with
mlen.Don't take this as a criticism, but I'm dubious about where you learned to use pointers. This is pretty much a royal hash. If you learned this from an online tutorial, example, or article, I'd strongly recommend avoiding it in the future. (Check to make sure you didn't just misunderstand first.)
The reason for the segfault is that wild pointer in #2. It's looking in the wrong place for a value, and it isn't finding it. You're lucky it failed with a segfault; it could have read in literally anything that was stored at that address in memory, whether it was coincidentally correct, subtly wrong, or absolutely bonkers. These sorts of bugs are really unpredictable like that.
Here's the operations you're misunderstanding in this code:
Getting a Pointer
To get the address of a value, for storing in a pointer, use the
&operator:Getting the length of a c-string (char array)
By the way...
One more pro-tip: stop using the
using namespace std;trick. It's an antipattern; in production code, it's all too easy to lose track of what-comes-from-where. Namespaces exist to reduce that confusion, butusing namespacenegates that namespace. Most examples use it for brevity; good production code never does.Instead, explicitly spell out namespace each time:
Save yourself weeks of headaches and refactoring now. Never use
using namespaceagain.Thanks for your help!
I think it works now :p
Excellent!
You will want to get into the habit of always testing your code thoroughly in Valgrind and addressing everything it complains about; even memory leaks. Undefined behavior has a habit of hiding until the most inconvient and unexpected moment. Write tests for your code (you should be doing that anyway!), and then execute those tests both outside of and within Valgrind.
Hey i write a program about TSP ant colony optimization and i get a segmentation fault when i compile and run the code, but when i debug it through gdb the program just runs flawlessly. What am i missing ?
You are dealing with what is called a Heisenbug, which is a bug, usually undefined behavior, whose behavior disappears when using debugging tools.
The first thing you should do is run the program through Valgrind (
valgrind ./myprogram). Ideally, you should do this on the Debug version of your program (compiler flag-g). This may provide you information on what memory errors exist in your code, and where they are in the source. Fix everything Valgrind complains about.However, if after doing that, you're still segfaulting, and even Valgrind can't pick up on any more errors, you're in for a bigger fight.
Start by reading my popular Stack Overflow Q&A Definitive List of Common Reasons for Segmentation Faults. This will attune your programming sense to what to look out for.
(I didn't include my personal favorite in that list: lambdas returning references can cause some particularly nasty undefined behavior.)
If you have an idea of when the segmentation fault occurs functionally, that can help you figure out what function(s) may be involved. If you can, try to create a Minimum Reproducable Example that has the segfault.
Print off the problem area of the code on paper. Desk check it with a red pen and a pad of paper. This means you act as the compiler, running the code mentally, and noting the value of each variable. I've caught a number of bugs this way.
If you're desperate, you can run the Release target of the program through Valgrind, although this will give you raw memory addresses instead of line numbers and file names. If you're very clever with a disassembler like Nemiver, and know how to read assembly code, you may be able to work backwards to isolate the problem. However, this is extremely hard; it will help a lot if you can do this with your Minimum Reproducible Example instead of the full program.
Good luck!
I think I kinda located the problem but i cant understand why is this happening. As you can see at the image above i for some reason decides to be whatever value it want's despite the fact that it is in a for loop.
thepracticaldev.s3.amazonaws.com/i...
This means it is reading from uninitialized memory. Common reasons for this:
You declared a variable, or dynamically allocated memory, but never initialized the memory with a value.
You are using a pointer (or reference) to either a position in memory which has already been freed (dangling pointer/reference), or which has never been allocated (wild pointer/reference). This can happen with either the heap or the stack; it's not limited to dynamic allocation.
You are exceeding the boundaries of an array or string (buffer overrun).
Have you ever worked with the Motorola 68000. I really like that CPU. In your opinion do you think assembly language is still best for super low level hardware or do you think C is on par with assembly code?
Ironically, I just added 68K Assembly to my list of languages to learn soon! I have a TI-89 calculator (Motorola 68000), and dearly want to play with it.
Up to this point, my assembly work has been largely limited to the X86 and X64 languages, in the context of Intel and AMD processors.
C is actually further up the stack than people think, and it isn't always the best choice for a given architecture. If you need total control, Assembly will always give that to you far and beyond any other language.
However, Assembly is also a pain in the butt (if an endearing one to certain classifications of nerds such as myself). If you have access to a higher level language that is reasonably optimized for that platform, and you don't need ultimate control, use it instead of Assembly.
In other words, "just because we can doesn't mean we should." If you can't make a reasoned argument for the language you're using, you're probably using the wrong language. :)
Thank you for the reply, I always value getting a second opinion. The reason I'm asking this question is because I am building a game on the Sega Genesis and I've been using A C compiler to do it.
So far it hasn't been an issue because the C compiler was built for the Sega Genesis and it has a lot of nifty features to take advantage of the hardware features such as DMA. More importantly it has sound drivers which are incredibly useful because I do not want to go around writing my own Sound Driver because I am not experienced with writing such a program.
I have recently run into a few short comings with the compiler. First and foremost being that the routines I've written in C don't seem to load as fast onto the screen as compared to Assembly.
I think I will compromise by writing my screen drawing routines in Assembly and then including them in my C code. I think that would be best for me because then I would have access to features in the C compiler as well as having access to the speed of Assembly. The problem is that I am not experienced with Assembly code. Fortunately for me, 68k assembly seems to be the easiest Assembly to learn.
By the way the C compiler I'm using is called SGDK (Sega Genesis Development Kit)
What do you think about mixing languages, is it something to be avoided?
It really depends on the languages!
There's no trouble combining C and Assembly; ultimately, C is compiled down to Assembly, at which point any Assembly code you wrote outright is just inserted in. Then, the whole thing is assembled down to binary on that particular platform.
However, you can run into varying degrees of performance issues when mixing other languages. It has to be taken on a case-by-case basis.
Bravo on making a game for Sega Genesis! Keep us posted on dev.to how that goes.
I highly recommend picking up "Game Engine Architecture" by Jason Gregory. It addresses many of the issues you're facing, and hundreds more besides, from a C and C++ perspective. He even talks about console development.
Hi Jason,
If we encounter a segfault with error code 4 or any other such error code -
localhost kernel: [139154.090095] xxxxx_process[11909]: segfault at 21 ip 00007ff5704b5254 sp 00007ff556bbbb98 error 4 in libmpi_global-release.so[7ff56c832000+6eee000]
However we see no assertions/errors and no core dumps been generated, how do we go along to debug such issues ?
Core dump configuration been verified and is correct -
Limit Soft Limit Hard Limit Units
Max core file size unlimited unlimited bytes
also the flag to generate full cores is been enabled !!!!
In almost all cases, it's very hard to debug a segfault without compiling the code in question with debug symbols (
-g) and running it through Valgrind.Core dumps are just snapshots of the raw memory when the program crashed, and will seldom provide any clues unless you are very familiar with the entire raw memory layout of your program.
Thank you for your response !!!
The problem here is we are facing this issue specifically in our client environment, we cannot reproduce this issue in house to try compile our code using -g or use valgrind by attaching it to a process
Using valgrind would add performance overhead in customer environment, so it's not a viable option, any other means to track this live on a client environment for a particular process ?
Off the top of my head, I don't know of any practical ways to debug a segfault in production like you describe. You could use logs and observations to determine what behavior(s) precede the segfault, and use that to focus in on part of your code base.
Meanwhile, your best bet would be to try and isolate what's different about their environment versus your test environment, and try and replicate it.
In any case, this won't be easy. This roadblock you're running into is exactly why it is so often said "if we can't reproduce it, we can't fix it".
Thank you Jason for the advise..
Will see if we can try to identify a diff in production env and in general..
Logs weren't much helpful to logically conclude in this case
I'm not as well acquainted with assembly as I would like yet, but here's my first thought: are you absolutely certain of the size (in byes) of memory addresses on your machine? This looks like you're blowing past the end of program memory.
Yes, I'm sure. However, I made some progress:
in the original 32 bit code, I had an instruction like:
that would work on macos (32 bit)
I found a 64 bit port somewhere that was instead using
which is what I expected, but the apple clang assembler does not like this syntax because in 64 bit mode I have to use position independent addressing modes.
So I tried
but it seems that derefences cold_start instead of just putting the address in. Using $cold_start(%rip) gives errors.
I guess I just don't understand the apple assembler syntax esp for 64 bit code. Looking ...
Given that you're writing a fairly low-level, perhaps wait-free concurrent algorithm, it's possible that you have a bug that trips up about every millionth execution of the code. It's a race condition that corrupts a vital structure. Any attempt to trace upsets the condition leading to the error, thus making it go away. Your only choice is a tedious hand execution and logical reasoning.
My question is, how do you avoid throwing the computer out the window?
Avoid? I love tedious hand execution and logical reasoning! (No, seriously.) One of my absolute favorite things to do in programming is to print off the source, sit down with a pen, a hot beverage, a blank notebook, and a jazz soundtrack...and then spend the next hour or three just desk-checking the entire thing.
Mmmmmmmmmmmmmmmmmmmmmm, bliss. ,^
Why do you think I specialized in memory management and undefined behavior? I ADORE it!
Now, if you don't have my particular mental condition, and actually don't enjoy desk-checking for Heisenbugs, my advice is this: get off the computer. Print off the source, cozy up in your favorite chair in a relaxing environment, and desk-check it.
I also recommend writing unit tests that makes the race condition more likely to happen. For example, if the code normally runs with < 10 threads, test it with 1000 threads. Sometimes code is well-behaved when the data entered are far apart, so try testing with consecutive values. If it's the opposite, test with random values.
What I learned over the years is that race-freedom is not composeable: code using several mutexes incorrectly could still suffer race condition, even though a single mutex is race-free on its own. When testing wait-free algorithms, start with very small primitives and gradually add onto it. And write plenty of assert() on the non-volatile local variables of the shared volatile variables the code might be using. When assert triggers under the debugger, you'll be able to see which invariants are violated in that snapshot.
Hi there, I am pleasantly surprised to discover this site.
Some people using my code had segmentation fault, I'm looking for a way to generate console output these people can copy and send me, so I can track down the issue.
When SEGFAULT happens, game is over. I'd like to capture this, inspect some variables and cout them before exiting, something like try catch. But I believe cout after SEGFAULT leads into undefined behaviour, so...
Any suggestion? Thank you.
Unfortunately, it is not possible to "catch" a segfault, nor continue program behavior safely (if at all) after it has been raised. Therefore, you have to take the opposite approach, and log everything that happened leading up to the segfault.
You can also have your tester describe (or screen record, especially if it's a game) what happened leading up to the segmentation fault. Then, you should be able to replicate that on your own machine.
Mind you, "replication" won't necessarily mean you can recreate the segmentation fault itself, since it's one of an infinite number of possible behaviors in response to some illegal memory action your code is taking (ergo "it is legal for the compiler to make demons fly out of your nose"). That's what it meant by undefined behavior. However, by replicating the same steps as your tester while running the Debug build of the application (compiled with
-g) under Valgrind, you should be able to catch the problem.There is also a more proactive approach you can take, especially if you're using C++: modernize your code base. Refactor the code - by hand mind you, NOT by using find-and-replace or some other automated tool - to make use of smart pointers like
std::unique_ptrandstd::shared_ptrinstead of raw pointers,new, anddelete. This will eliminate most memory errors, since the smart pointers handle object lifetime and whatnot (formally known as RAII). Refactoring is not a "quick fix", but it's the most resilient fix.Thank you very much. You confirmed my approach is right: cout everything!
In my specific case, my code is appended to third party code where the segfault happens, so it's hard to trace and I don't even have the chance to fix.
At the risk of self-promotion, I wrote something called IOChannel which is designed to better control
cout-style logging, based on category and priority. You can also route messages to different places, including to functions that will write them out to a file instead of printing them to the console. It's part of PawLIB, which is still in development, but 1.0 is stable. (Yes, totally open source)Hi Jason, pleased to meet you.
I'd like to present you a question about C memory management, if you were so nice. In short, it's about a program that lets you input or random generate a set of 5 numbers between 1 and 50, and 2 numbers between 1 and 12 (EuroMillions lottery). Then it can keep rolling over and over (more than 139 million times on average) until it gets the same set of numbers. The issue is that, despite using dynamic allocation and de allocation for structures, as code executes it would eventually exhaust the RAM memory until the process gets killed by the system to avoid stall.
I've tried a lighter version of the code (just 5 numbers from 1 to 48, 1.7 million loops on average) with no problems, and ensured the code is actually recycling the memory used with each pass. Any ideas of what could be wrong with it?
Thanks in advance, regards.
I'd really need to see some code to be able to debug this, but here's the first two things I'd look for:
Double check that things are actually deallocating; you'd be amazed at how often one thinks they have free'd memory when they haven't. You may be able to run it through Valgrind or another dynamic analyzer to check that. (It sounds like you've done that, though.)
You may have some other variable you didn't think about, either on the stack or on the heap.
If it wouldn't be too much trouble, can you put the code in a GitHub Gist or another paste bin? I might be able to catch the problem better if I read it.
Thank you so much for your quick and kind response Jason! I will paste the code to GitHub and share here a.s.a.p, I'm a newbie to it, as well as to many other things. Will take your advice and check out what you suggested.
We'll keep on touch.
Regards!
Hi
I have a multi threaded C++ application. Its a read write case i.e. T1 write/mallocs a char* and T2 reads and takes actions according.
Can relloc result in segfault? I donot want to use free as that can result in segfault when the other thread tries to read. Does relloc also comes with the same uncertainty for segfault?
First, the deeper problem: if you're using C++, why are you using
realloc? This is C-style memory management, which certainly has its place, even in modern C++ code...but very very rarely. In almost all cases, it's better to usestd::shared_ptrorstd::unique_ptr.That said,
reallocis perhaps one of the few reasons to use C-style allocation, but only if you need it for optimization purposes. (Remember, premature optimization is the root of all evil!)For the rest of the answer, I'll assume you have a very good, well-reasoned argument for using
reallocand C-style memory management in C++. If you don't, stop right now and rethink your code in terms of modern C++ memory management. It's treacherous usingfree/malloc/calloc/realloc, and evennew/delete, without having a very clear and well-defined architectural reason.realloccomes from the C language, so the best way to find out what it does, and if there is any potential undefined behavior, is to check the official standard.(You can purchase the official standard, but if you're anything like me, you don't want to drop nearly a hundred quid just to read the thing; you can just read the final draft instead, which is pretty much the same but for a few minor corrections. You can find the official PDF here.)
So, here's a few important pieces from 7.22.3.5...
There's the answer to your first question.
reallocwill free the old pointer, the same asfreedoes. Thus, if you access/dereference that freed pointer after callingreallocon it, the behavior is undefined. A segfault is just one possible form of undefined behavior.This is also an area of potential weirdness! If you realloc with a larger size, the new space might contain literally anything. In other words, if you start with an array of four integers, and realloc it to an array of eight integers, indices [3] through 7 is not guaranteed to be set to
0, but could be anything. Thus, you'll have to be careful to initialize the new values in those spaces before reading from them.From this, we know that passing a null pointer to
reallocis not going to be a problem; it'll just behave likemallocwould. Good to know.Don't pass a dangling or wild pointer to
realloc; that would result in undefined behavior, such as (but not necessarily being) a segmentation fault.This is interesting! If you pass a pointer to
realloc, but it cannot reallocate for any reason, it will not free the pointer. How do we know if it could reallocate? Read on...If there's a problem allocating,
reallocwill return a null pointer, indicating that the pointer we passed in was not freed, and thus is still valid and safe to use.Also, important note about assumptions here: the new, reallocated pointer might actually be the same as the old pointer. Or it might not.
All that said, you should use
reallocwith the following assumptions:realloc.reallocwill befreed.reallocreturns a null pointer, it couldn't reallocate, and you can keep using the pointer you passed in (it was notfreed.)reallocreturns a pointer, it's valid, regardless of whether it seems to be the same as the one you passed in.Thanks for the response. I solved the problem in a very simple manner.
Your detailed reponse really helped me think in the right direction.
Hi Jason, I'm encountering a strange seg fault when trying to write default values to an mmap'd region on my disk, but when I try to write to the file manually in gdb, it comes back normal. Here is what I'm talking about
So when I do it myself, there's no segfault. I've tried to run valgrind, but it's so slow, can you take a look at it? The code is uploaded to github at github.com/iggy12345/reversi_walke...
I've been working at this one for days now and I'll take any help I can get.
I would run it through
valgrind. That should take you right to the line of code that the segfault is being thrown at. (You're welcome to share that output here.)gdbrarely provides much useful information for undefined behavior.I'll try and see if I can get it to finish, the segfault seems to happen at 18mil final board states, and at 3mil/sec it takes about an hour and a half to get there, but with
valgrind... I'm an hour past 1 day and I'm still only at 3mil boardsAlthough it's an early (and wild) guess, if the segfault occurs with a large set of data, but not a small set, I would suspect a buffer overrun may be the cause of your problems. Are you...
(1) Putting too much on the stack (versus dynamically allocating the space you need), or
(2) Exceeding the space you allocated?
Hello, Jason
I work in a house code that is able to simulate fluid dynamics. Nowadays it has four hundred thousand of lines. An important piece of the code uses PETSc library that can solve linear system. The code uses MPI for parallel communication, C language and Fortran, but most of them are in Fortran90. After new version of GCC (>8.0.0), the code started presented a memory leak, when the Petsc library is active. I tried to use Valgrind and DrMemory to get that leak, but I still could not find the problem. I'm quiet sure that the problem is not in the Petsc Library, but in the way that I communicate the global matrix. I noticed that you are expertise in memory leak and perhaps you can see the valgrind log and help me to detect where the memory leak is happening.
In valgrind log there is a Lot of information, but all of them point to the MPI library or HDF5 library, but any one points to the f90 files or Petsc Library. In that situation is possible to get the memory leak?
the way that I run with valgrind:
mpirun -n 2 valgrind -v --leak-check=full --show-leak-kinds=all --log-file=valgrind%p.log --track-origins=yes ./amr3d
Thanks in advance
Millena
Hi Millena,
Of course, without seeing the Valgrind log itself, it's hard to say. (If you do post that, please use a GitHub Gist, a Hastebin, or something of the sort.)
First, some technical background. Apologies if you already know some/all of this. It's also for other readers:
Memory leaks can be an issue, but they aren't necessarily a sign that something is wrong. In some particularly complex programs or libraries, it is impractical to manually free all of the allocated memory at the time of program termination, so it is acceptable to just allow that memory to be freed as a part of the entire program's stack and heap being released. So, in that sense, it is perfectly possible that a memory safe program can report memory leaks.
However, as you know, a memory leak can become an issue if it is occurring during the program's lifetime, instead of just before termination.
In any case, a memory leak always has the same cause: memory is being dynamically allocated, but not freed before the last pointer to it goes out of scope. Thus, it becomes virtually impossible to free said allocated memory, so it can never be reused during the life of the program. If this happens enough times, you can actually run out of heap space.
One more issue that makes this difficult is that a memory leak can occur in one place, but be caused by usage elsewhere. For example, you might call a library function that allocates memory, but not realize you must call another library function to deallocate it. That's far more likely to occur in C, instead of C++ or FORTRAN, due to C's lack of objects and their constructors/destructors for handling automatic allocation and deallocation related to the lifetime of an object. That's why the entire stack trace is so important.
Here are my initial thoughts on actually tracking this down.
First, I find it interesting that the problem only occurs after you start using a new version of GCC. This does not necessarily mean there's a bug in the compiler, however. Memory-related bugs have a tendency to develop strange properties, such as Heisenbugs or Schoredinbugs. There are many more such freaks of nature besides.
I wonder if the conditions for a memory leak have been present in either your project's source or the library source for some time, but that particular implementation details (or another bug?) in previous versions of GCC concealed its existence. Once the behavior of GCC or the standard library were changed in the latest version, possibly fixing a bug, the memory leak was no longer being coincidentally diked out. In fact, I'd wager this is the most likely possibility.
I'd also suspect that the memory leak itself actually is in the Petsc library, given that it must be involved for the leak to occur. It may even be possible that the latent bug existed in Petsc. However, the cause of the memory leak quite possibly originates from your source; your particular usage of the library may be triggering some sort of peculiar corner case in Petsc, wherein the bug resides.
The other possibility is that GCC 8 has a bug itself, but given the size and domain of your program and its libraries, that would be quite difficult to isolate.
And now the bad news: tracking this down is probably non-trivial. If you compile both the library and the source with
-g, Valgrind should give you line numbers you can use to check the source. You'll need to work backwards to figure out what's wrong.That would be the easy solution, and hopefully it's as far as you need to go.
I would also recommend testing your code against the LLVM Clang compiler, if possible. Does the same error occur there? If it does, be glad! You need only pick apart your code and that of Petsc to find the problem.
Otherwise, if GCC 8 does prove to be a necessary environmental factor for the memory leak to occur, and you cannot isolate the problem any other way, another rather involved thing you can do is to perform a bisect on compiler versions.
Spin up a clean environment, such as in a virtual machine.
Build your libraries and source with the last compiler version you remember working. Ensure the memory leak is not present.
Build with the compiler version you know isn't working. Verify that the memory leak is present. (If it isn't, you can rule out compiler; you're now probably dealing with a phase-of-the-moon bug, which will require you figure out what on your development machine is causing the issue.)
Assuming 2 and 3 have the expected outcomes, use those two compiler versions as your endpoints for a bisect. Check compiler versions in between, following the same workflow as a
git bisect, until you know the first version that presents a problem.If you have the time and access, consider bisecting on the development versions leading up to the first version of the compiler that presents the issue.
Check the changelogs for the version. If you did step 5, look through the commit messages. Try to isolate the change to the compiler that is contributing to the memory leak.
If you do all this, remember, this might not be an actual bug in the compiler. If you can determine what caused the behavior to change in the library, you may be able to find the latent bug in Petsc.
I hope that helps!
Hello, Jason
First of all thanks a lot for you opinion and advices!. I will try to perform step by step what you suggested above. In sequence there is a link where the valgrind log is. As the tests are being performed I would like to share with you the results. Can I have a contact with you by email?
gist.github.com/mmvillar/ca0a726a4...
again, Thank you!
Sure, my contact info is on my personal website. Link is on my DEV profile. I can't guarantee that I'll be able to solve this remotely, as you know the code far better than I could hope to, but I'll help where I can.
EDIT: Looked at the log. Yeah, you'll definitely need to compile your dependencies with
-g, in order to have all the information you need.I know this question is going to come so I'm going to ask it myself: what do you think of languages like Rust?
Do you think that, in some cases, isn't a machine going to be better than a human at dealing with memory management anyway?
See the other question in this thread re: Rust. Long story short, it's a cool language that I haven't yet had time to learn.
In terms of man vs machine, the answer is that "computers are inherently stupid." When we are trusting the machine to manage the memory, what we're really doing is trusting someone else's code to manage the memory. In either case, some human is responsible for the memory handling logic. Therefore, it really depends on the code you're trusting!
The benefit to trusting the language's built-in memory management is that the code is almost certainly more rigorously reviewed and tested. That's where the apparent added trustworthiness comes from.
A lot of times, I will trust automatic memory management tools over my own abilities.
std::unique_ptrandstd::shared_ptr, for example, are excellent tools that help minimize memory mistakes (because, after all, I'm only human). However, there are times that the logic I need would become too convoluted with those magic pointer classes, so I'll resort to manual management.It's basically a balancing act between simplicity (the more complicated the code, the more chance for bugs) and safety (reducing the chances of a memory leak). If you write really complicated code to use "memory safe" tools, you can still wind up making a royal hash of it, when a simple pointer would have meant 80% less logic, and thus prevented those issues.
Thanks for the detailed explanation!
I wonder if in the future there will be attempts to introduce AIs into managed memory systems, to increase what you call "apparent added trustworthiness".
Oh what a small world. I too am an expert in segfaults. Nary a day goes by without my code segfaulting...
Sorry, obvious joke that I didn't see anyone else make. Thank you for doing this, it's very insightful.
That's how you become an expert at solving them. ;)
hey! I have been asked to perform basic arithmetic operations on linked lists.. my program can add no.s upto 50 digits.. and multiply two numbers of 25 digits max. But after that i am getting a seg fault (core dumped) . Can you help me to improve the performance.. linked lists are performed as traditional methods.. nothing new
I can try, but you'd need to share some source code.
To provide maximum visibility, in case someone else is in a better position to help than I for any reason, I recommend creating a new post for your question, and including the
#helptag. Then link me to it here. :)Hi Jason !
I am trying to get valgrind running on cavium octeon 3 processor.
Couldn't find any specific build configs for this other than some hints on the valgrind wiki.
The steps executed to build valgrind for cavium were (on x86_64 host):
export CC=/opt/cavium-64bit/tools-3535/bin/mips64-octeon-linux-gnu-gcc
export CXX=/opt/cavium-64bit/tools-3535/bin/mips64-octeon-linux-gnu-g++
export PATH=$PATH:/opt/cavium-64bit/tools-3535/bin/
./configure --host=mips64-linux-gnu --target=mips64-octeon-linux --prefix=/home/mihira/workspace/mylab/valgrind/cavium --exec-prefix=/var/tmp/cavium --disable-tls
make && make install
While trying to run it on target (copied the elfs to /var/tmp/cavium/ dir on the target):
export VALGRIND_LIB=/var/tmp/cavium/lib/valgrind/
export PATH=$PATH:/var/tmp/cavium/bin/
valgrind -h
-bash: /var/tmp/cavium/bin/valgrind: No such file or directory
What's that blunder I am doing?
Hi Jason! I've been having some trouble with an open source computational chemistry program, maybe you can help. The main program is called Enso (github.com/grimme-lab/enso), but I'm specifically running into the issue with one of the resources included with it. It's the statically compiled binary called anmr. I'm running windows subsystem for linux, Ubuntu 20.10. My PC has 32 gb ram and a Ryzen 7 4800H processor (8 cores).
The issue:
Every time I call the program, even with 'anmr -h' just to bring up the help menu, I get the following output:
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
anmr 000000000180DB13 Unknown Unknown Unknown
anmr 00000000019E2130 Unknown Unknown Unknown
anmr 000000000040A133 Unknown Unknown Unknown
anmr 0000000000403542 Unknown Unknown Unknown
anmr 00000000019E35B0 Unknown Unknown Unknown
anmr 0000000000403427 Unknown Unknown Unknown
What I've already tried:
I'm not the first one to encounter this error, however, everyone else was able to solve it by unlimiting stack size with "ulimit -s unlimited". The short version: it didn't work for me. The long version: I couldn't initially use the command because I didn't have permission to change the stack size hard limit. I managed to change that by altering files such as /etc/security/limits.conf, /etc/pam.d/common-session and so on. When I initially open the terminal, the limit changes don't take effect, but they work when I use "su vilsenhet" even though that's my default profile when I open the terminal. After unlimiting stack size, core file size, etc, calling anmr gives the same segfault error. I reached out to the original programmer for help, and he checked for errors and recompiled the binary for me. It didn't fix the problem. I've tested the archive with 7zip which found no errors. I've added every directory I could think of to my path and LD_LIBRARY_PATH. I've tried a few other longshot fixes I've found by googling the problem, but nothing helps. I'm at a loss.
I'm pretty new to linux and to programming in general, so I really have no idea, but I'm thinking maybe the problem arises from the fact that I have to switch user after my initial login to be able to alter the ulimits, so somehow they don't actually apply to the program when I call it? I don't know. Any insight, tips, or tricks you could offer would greatly be appreciated. Thanks!
First step w/ these is always to compile the program with debug symbols and run it through Valgrind. It's almost impossible to track down the source of the problem without knowing where in the code the program aborts.
Hello Jason, zmq-bind-proxyd process crashed. The backtrace is mentioned below. As per the stacktrace, crashed originated from zmq_poll function call. I do not see any reason why process would crash at line 643. The instruction is not dereferencing either NULL or uninitialized or dangling pointer. no buffer overrun or stack overflow. Can you please help me find root cause for this ?
643 if ((rc = zmq_poll (in_items, ARRAY_SIZE(in_items), -1)) < 0) {
644 sd_journal_print(LOG_ERR, "%s: custom_proxy - ERROR! (%s) polling-in",
645 name(), zmq_strerror(rc));
646 break;
647 }
Program terminated with signal SIGSEGV, Segmentation fault.
warning: Unexpected size of section `.reg-xstate/346' in core file.
0 0x00007fe7311ef9cc in ?? () from /neteng/nagaraj.venkatapuram/OS10_repo.f10/10.5.3.2.99999/usr/lib/x86_64-linux-gnu/libzmq.so.5
[Current thread is 1 (Thread 0x7fe72ca40700 (LWP 346))]
Setting debian release code name to 0
Sourcing gdb user defined commands
warning: /home/aruljeniston/gdb-macros/os10/zmq_gdb.gdb: No such file or directory
Redefine command "libevent_active_list_dump_by_flags"? (y or n) [answered Y; input not from terminal]
Redefine command "libevent_walk"? (y or n) [answered Y; input not from terminal]
Sourcing miscellaneous gdb-macros
Sourcing dn_sm gdb macros
--------------trace-----------------
0 0x00007fe7311ef9cc in ?? () from /neteng/nagaraj.venkatapuram/OS10_repo.f10/10.5.3.2.99999/usr/lib/x86_64-linux-gnu/libzmq.so.5
1 0x00007fe7311d314c in ?? () from /neteng/nagaraj.venkatapuram/OS10_repo.f10/10.5.3.2.99999/usr/lib/x86_64-linux-gnu/libzmq.so.5
2 0x00007fe7311d3814 in ?? () from /neteng/nagaraj.venkatapuram/OS10_repo.f10/10.5.3.2.99999/usr/lib/x86_64-linux-gnu/libzmq.so.5
3 0x00007fe7311f40ea in zmq_poll () from /neteng/nagaraj.venkatapuram/OS10_repo.f10/10.5.3.2.99999/usr/lib/x86_64-linux-gnu/libzmq.so.5
4 0x0000564cb2ace363 in zmq_proxy_c::custom_proxy() () at zmq-common-proxyd_zmqctx.cpp:643
5 0x0000564cb2acece7 in zmq_proxy_c::c_proxy(void*) () at zmq-common-proxyd_zmqctx.cpp:529
6 0x00007fe730cf2fa3 in start_thread (arg=) at pthread_create.c:486
7 0x00007fe730c234cf in clone () from /neteng/nagaraj.venkatapuram/OS10_repo.f10/10.5.3.2.99999/lib/x86_64-linux-gnu/libc.so.6
(gdb
About Garbages Collectors (not mentioned so far !) i think that they are nice but should not be a default management so rather optional. Once a GC handles the allocations it's very hard to use another alternative management on top because GCs tend to free manually allocated resources since they think they are not used anymore, i.e not used by a root memory block.
What's your favorite management technique: manual, ref counting or GC ?
Do you share my point of view on ?
Since I use C++ primarily, and it doesn't have a built-in garbage collector by default, I've just formed the habits of handling everything myself. Those habits and instincts carry over to other languages that do have GCs, but I will still manually free things as far as I'm allowed.
In a broader sense, I don't generally trust generic abstractions to do my work for me. If I'm not sure what's needed, I'll leave it to the automatic systems, but try to understand what's happening under the hood. If I know for certain what needs to happen, I'll do it myself, and let the automatic systems do mop-up work behind me in case I miss anything.
In the same way, I never let the compiler define constructors or destructors for me. Every (non-static) class I write has, in the least, explicitly empty constructors. Knowing how my coding adventures usually go, the one time I trust the compiler to define the destructor, it'd hit an edge-case and bork. So, I don't leave much room for that kind of madness.
Ironically, the above is probably in part my Python background talking: "explicit is better than implicit".
Now, with that said, one should know all the automatic tools their language offers, and how to use (and not to use) them. Doing things manually is not an excuse for ignorance. All the above does not preclude me from using such bits of magic as
std::unique_ptr, which handles its own deallocation via a GC. I simply make an informed decision on whether to do it myself, or to use a tool that specifically matches the use case.By the way, in terms of ref counting, I am reminded of a classic AI Koan...
hello can i have help please i have a segfault error : kernel: [69100.813469] ECM Thread[12862]: segfault at 59 ip 000055931d313a14 sp 00007fbb87ffe870 error 4 in multics[55931d29f000+ab000]
and i have debug with gdb core
his is the line with segfault : struct mg_client_data *next = cli->nextEcm;
can any one tell me how to solve this error please ?
here the db result :
dev-to-uploads.s3.amazonaws.com/up...
Hello,
Looking through your code there, it does appear that somehow
clion some iteration is a wild pointer —0x59, apparently, which is a rather weird pointer address — meaningcli->nextEcmis almost certainly returning a wild pointer at some point.The most common cause of a wild pointer is uninitialized data. You need to check the code that produces those structs you're traversing, and ensure the default value of
cli->nextEcmisNULL. C does not have implicit default values; when you declare a variable without providing an initial value, it will just use whatever value happens to already be at the memory location already allocated. Look for that first; ensure everything, especially your pointers, is initialized with some value, either a valid pointer orNULL.I'd also recommend running the whole thing through
valgrindanyway. It might fail at a different place, or at the same place with some additional information.Hello Jason, I wonder if there is a good pattern for callbacks to object methods (or in general callbacks that need to operate on object instances). There are multiple things I find quite ugly (the way I'm currently implementing it):
Is there a better way to do that in Cpp? I thought about implementing a functor template to wrap the two pointers. As attribute of the owning object, the pointers are no problem, since lifetime is automatic with respect to the owner. The functor could create copyable views of itself, which share the same pimpl with a weak_ptr, but hides the object type. The caller could then simply receive this view-functor and call it, passing only the necessary parameters and not knowing the the receiving object.
This might work well for my own programs, however what about callbacks passed to other libraries? I guess in this case there is no way around the problems 3. and 4.?
My mind is pretty deep in an unrelated topic, so apologies if my answer is less than ideal. I don't want to wait and forget to answer altogether.
There are two thoughts I have:
First, sometimes our "data hiding" instincts are a tad misplaced. I've done a lot of work in Python, which has little concept of data hiding, and yet everything works. If you design a class well, such that you clearly indicate the intended usage, then you can put the responsibility on the user of that class to not jump the clearly marked fences without knowing what they're doing.
That said, what you're describing is more along the lines of abstraction and loose coupling, rather than data hiding, and those are good concepts to embrace!
I like your thoughts regarding having an intermediary view-functor type object. A shared_ptr, or at least a weak_ptr that is checked before dereferencing, should resolve your concerns about a dangling pointer.
It's really hard to give very concrete advice here, though, as I have no idea what you're building from a practical standpoint. This feels like a really weird architectural situation.
It does remind me just a little of a function I designed that needed to call different functions based on traits of the first argument, and which accepted different required and optional arguments depending on those traits as well. Templates were incredibly helpful in that regard: github.com/mousepawmedia/iosqueak/...
Hello, Jason!
I'm trying to recreate the ls command, but catch the segmentation fault when trying to print recursively ls -R / command. I'm checking if I have permission to open the folder and it's working pretty nice. Each time it reaches different depth of file tree. Could you, please, give a piece of advice how to handle it and what can cause the segmentation. Thanks in advance!
The first rule of debugging: "If it's weird, it's memory."
You have to remember, a segmentation fault is just a form of undefined behavior. You can use a dynamic analyzer, such as Valgrind, to dial in on the exact part of the your code with the problem.
Look especially for the following "hot spots" for undefined behavior:
A while back, I wrote up a big list of common reasons for segmentation faults. It might be helpful.
Hello Jason,
I have a fairly big code written in c++ (not by me).
The code has been working fine for the most part, until I upgraded the system where it is running to a newer version of debian (bullseye).
We are getting a segfault when one specific operation of the code is executed.
I get the segfault with the code compiled in both optimized and debug flavors.
However, when I run the code with either gdb or valgrind, the code works fine. Valgrind doesn't show any errors.
The last thing I tried was running the code without gdb/valgrind and generating a core dump. Opening the core dump with gdb shows that the segfault happens when I call realloc(), so apparently something is messing up that pointer when it is freed. I tried replacing the realloc by a malloc() followed by a memcpy() (without freeing the previous pointer), and it worked fine.
I was reading in other blogs and it might be that both gdb and valgrind change the memory layout of the code so the bug doesn't show up, so if that's the case, how can I track it ?
Any suggestions would be greatly appreciated!
Thanks a lot!
F.
Ahh, the joys of a Heisenbug. Unfortunately, you seem to have gotten a particularly shy one. There won't be a way to directly observe it. However, your clever investigative efforts with the core dump have pointed you to the responsible
realloc(), and thus the pointer giving you trouble, so there really isn't much more that Valgrind could have told you.Thinking through it, I wonder if the free is causing issues because the memory at the original pointer was partially or completely uninitialized? Desk check the code allocating, initializing, and accessing that pointer, preferably in execution order (or close to it). Remember that a segfault is an operating-system-level error raised because something attempted to access memory not belonging to the program - or, more specifically, trying to use a memory address in the "protected space" that the OS set aside around the memory assigned to the process.
I hope that helps.
Hello Jason,
Thanks a lot for the reply and the suggestions.
This is the gdb/core dump session:
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `executable'.
Program terminated with signal SIGSEGV, Segmentation fault.
0 0x00007f629d991df8 in mremap_chunk (p=0x7f62938d3000,
p@entry=0x7f629960c000, new_size=, new_size@entry=3787904)
at malloc.c:2878
2878 malloc.c: No such file or directory.
(gdb) up
1 0x00007f629d996a70 in GI_libc_realloc (oldmem=0x7f629960c010,
bytes=3787896) at malloc.c:3206
3206 in malloc.c
(gdb) up
2 0x00007f629c37cadb in ?? ()
from /usr/lib/x86_64-linux-gnu/libnvidia-tls.so.470.129.06
(gdb) up
3 0x000056491c746ab6 in MyClass::allocateF (this=0x56491d35bdf0, n=157829)
at program.cpp:46
46 xyz=(double*)realloc((void*)xyz,3*n*sizeof(double));
(gdb) q
It looks like realloc() goes into the library libnvidia-tls(??) which is strange (at least for me).
So to test things out I created myrealloc() in a different .cpp file, by just doing malloc, memcpy and free and the segfault disappeared although I am not sure if that really solved the problem or if I just pushed the bug away for now.
My first thought was that there is a mismatch between the malloc() that creates the pointer in the first time (maybe from a different library than libnvidia-tls) and the realloc() that is being called here (as it is fairly easy to redefine realloc() with a macro). However I think this would have also generated a segfault within gdb/valgrind.
Have you seen something like this before ?
Thanks again!
Is there any technique to deliberately cause a segmentation fault and analyze? May be for example, to understand the code-flow in multi-threaded application, probably under a specific application state?
Dereferencing a null pointer is usually a pretty reliable way of causing a segmentation fault. Of course, in so doing, you're killing the program. You could just as easily use assertions or system calls to deliberately crash in a more controlled manner.
C is my favorite language, but I have never worked with it professionally. I've completed "Learn C the Hard Way".
What's another challenging text for a professional developer who is a C dilettante?
I've got a few on my shelf I enjoy in that category:
Game Programming Patterns by Robert Nystrom discusses many patterns, including a number related to dynamic allocation and memory management, from a game development perspective. (Written mainly for C++, although you could take on the challenge of implementing the patterns in C!) Besides that, his comical, bantering style makes for a really fun read.
Hacker's Delight by Henry S. Warren contains a number of mind-bending algorithms that operate in C and Assembly.
Game Engine Architecture by Jason Gregory explores the myriad of challenges that game engine developers face, especially issues of performance and memory management. Again, this written primary for C++, but you can approach many of the problems from a C perspective as well.
The Art of Computer Programming by Donald Knuth. Okay, I don't own this one, but I really really want a copy! It's quite a challenge to wrap your head around his algorithms and patterns, many of which are fundamental to the field of computer science.
Hello , If you could help me , it's a school project the thing is , I get segfault when I run and type a big string as an input now the real thing is I have a buffer big enought and when I run with valgrind I get no seg faults do you have any idea?
I could probably help, but I'd need to see your code to do so. Can you create a Github Gist?
what’s the simplest way to cause memory leak ?
Fail to free memory after you allocate it, and then destroy and the pointer.
My return question is, why would you want to? ;-)
Ironically, you of all people should have a very easy answer to your own question -- 'Why would you want to?' -- To learn more about what went wrong? To study it, to understand it, and to prevent it from happening again in the future. Sometimes, the best way of learning is by knowingly doing something 'silly' -- I wouldn't call it stupid because you are doing it with the expectation, which means you are preparing for it. All good engineers try to reach this state -- be aware of what can go wrong, and how to handle it.
Yes, that's fair, in learning. But, honestly, 99% of my learning comes from just trying to do hard things, and working with the failures as they come. Those are far more practical and effective to learn from than any sort of deliberately manufactured mistake.
Hi, Jason: THANK YOU for your kind offer. I am trying to develop C/C++ on WSL2 Ubuntu, then transfer code to a machine running straight-up Ubuntu . I'm writing/compiling C++, but using many C functions and features, notably realloc(). I get a "malloc(): corrupted top size\Aborted" error on my local WSL2 Ubuntu machine when I try to initially realloc() a variable larger than about 64B. But when I compile and run the same code on the remote machine running straight-up Ubuntu, (assuming the same gcc version, but I cannot be sure), I do not run into the error. My local machine, (running WSL2 Ubuntu) is much more beefier than the (remote) Ubuntu machine, (local is I-9 with 32GB RAM, and I believe the remote [school's] machine is a P4 Core2Duo with about 8GB RAM). Do you have any idea what is going on, or could you guide me where I should troubleshoot? THANK YOU AGAIN very much for any time or thoughts you can spare.
So the remote [working] gcc is version 7.5.0 on Ubuntu 18.04.
My local [error/aborting] gcc is version 9.3.0 on Ubuntu 20.04, (in WSL2).
Thanks again for any thoughts!
Hello Jason!
I was running a multi threaded code where I am getting a segfault at a particular line.
As shown in the image the curr_node in the code is a healthy node that I am able to check on my debugger.
Hi gaurav,
(1) I can't see the image.
(2) multi-threaded segfaults are scary. Are you able to try and reproduce it single-threaded? If you can, it'll be easier to debug. If you cannot, it may be related to the threading itself.
(3) This is probably a lot deeper than just finding a segfault. You will want to audit your code for thread safety.
Hello.
Can you take a look at:
stackoverflow.com/questions/645513...
I assume bug in my toolchain but I have no idea what can I check.
Regards
Jakub
What is your preferred method to return from a segfault?
i.e. define
If I understand your question right...
A segfault is the best possible behavior that you can get given undefined behavior, because it's a specific runtime error that you can probe. You're actually not guaranteed to get a segfault when your code has undefined behavior.
Thus, it is both technically impossible and entirely unwise to "recover" from a segfault. Let the program crash (did we have a choice?), figure out what in your code is undefined behavior, and fix it.
To put that another way, because a segmentation fault is a runtime error, and one that isn't guaranteed anyhow, it's immune to try-catch statements and error handling.
What do you think of Zig?
Any advice for complete new coding beginners please would appreciate it?
Sure.
Don't mess with manual memory management...yet.
It's very, very easy to proverbially blow a limb off with manual memory management. Get skilled with the fundamentals of programming first, and establish habits that allow you to write clean, stable code in a higher level language.
Once you can write a few hundred lines of code in, say, Python or Java, and have them work right on the first or second attempt, then dive into more advanced concepts. Manual memory management is something that's easy to get wrong, so you need to first have an established track record with yourself of getting things right.
Before I ever touched C++ and memory management, I had written a couple of reasonably stable, small applications, and had actually implemented a programming language in ActionScript 3.0 with regular expressions. (I don't recommend the latter; it was a great challenge, and it worked great for the purpose it was designed, but it pretty well sucked in terms of performance.)
With all that under my belt, I was able to start using C++. Even then, I avoided manual allocation whenever possible, using memory-safe tools and methods first. Once I was experienced with those, I started doing more and more manual allocation and raw pointer arithmetic. I made a lot of mistakes at the start, but that's how we learn best!
What are some of the common issues you find when working in memory management? Do you have one way of working through them or multiple?
I do go through a little list in my mind:
mallocand its cousins withfree,newwithdelete,new[]withdelete[]).whileloops that touch allocated memory.)Can I or can I not return and use an address that does not point to the start of a dynamically allocated number of bytes but somewhere inside the allocated memory?
Yes. As long as the pointer points to a sector of memory which is within in allocated region of the program's heap memory, it is valid, and can be accessed and used.
However, be aware that there can be unpredictable problems if you read from uninitialized memory — memory which has been allocated, but no value assigned to it.
How to create segmentation fault using malloc?
Segmentation faults are just a possible outcome of undefined behavior, which is unpredictable by nature. They are not like exceptions! There is no official or reliable way to "throw" a segmentation fault, as it only occurs with undefined behavior, wherein "it is legal for the compiler to make demons fly out of your nose". No matter how you "cause" a segmentation fault, there's always a chance it will do something else, or even somehow appear to produce valid code instead!
Meanwhile,
mallocdoesn't tend to have undefined behavior, unless you're dealing with heap corruption (a very bad thing), wherein you somehow wrote into unallocated memory. If that's the case, your problem isn't themalloccall, but rather something elsewhere. Barring problems from elsewhere in the code,mallocwill only do one of two things: return a pointer to allocated space, or return a null pointer because it couldn't allocate. Simple as that. :)Hello I hope you still alive I just want to ask you about :
-----How the segfault is represented on the memory thank you !
Hello, yes! Alive and well.
When an operating system works with virtual memory, it has to allocate memory addresses to a given page of memory. For example, it might allocate a page of memory to Chrome for anything it wants to allocate/deallocate (heap space). However, it also reserved a range of addresses (but not memory) which should not be accessed by anything. The purpose of this is to ensure that when you are iterating over addresses in that page of memory, you don't leave the page and cross into another. Instead, you hit those reserved addresses, and the operating system immediately throws a segfault.
Short explanation, but I hope that helps.