Intro
Like many of you, I am a software developer. For the past few years, I've been working with Python, both at work, and writing small hobby projects at home.
One of the most common things I do with Python is write Linux services/daemons. A linux daemon is a program, in our case written in Python, that runs in a loop, usually by SystemD, and only exits when it receives a kill signal.
A few months ago, I decided to teach myself Rust, and after reading the Rust book (which I highly recommend), and watching lots of youtube videos, I tried to write a Rust Linux daemon.
Part of the process was figuring out how to handle kill signals in Rust, which is very different than how it is handled in Python. I expected this to be relatively easy, but found it to be rather difficult. In this article, we're going to take a look at the signal_hook crate, and the different ways Rust allows us to react to a kill signal. Hopefully, my experience will help ease the journey for some of you.
Linux Kill Signals
The 2 most common kill signals in Linux are SIGINT, which gets sent to your program when you run the program manually, and hit Ctrl+C in your terminal, or SIGTERM, which SystemD sends to your program when you run systemctl stop my_program.
Unlike Python, which throws an ugly traceback when it receives an unhandled kill signal, Rust quietly exits. If your Rust program does not need to do anything special when it terminates, you can probably get away with not doing any kill signal handling at all, but if you need to do any kind of cleanup, you'll likely need to catch these kill signals, and do your cleanup before exiting.
Crate Options
Unlike Python, Rust doesn't have built-in kill signal handling. You'll need to use an external crate to add this functionality. There are 2 main crates that allow you to handle kill signals:
The ctrlc crate is fairly easy to use, but only supports a limited number of kill signals:
- SIGINT
- SIGTERM (requires enabling the termination feature)
Here, I'll be taking a look at the signal_hook crate. Once we break down the signal_hook crate, the ctrlc crate becomes easy to understand and use as well.
Signal Hook
This may change, but as of now, the crate documentation has 2 examples on the front page - one that is very simple, which we will take a closer look at below, and one that is overly complex, which uses features that are unlikely to be commonly needed, and won't compile without enabling extra features. We'll slowly work our way from the simple example, to a more complex one that resembles the one in the documentation.
Cargo.toml
The first thing you'll want to do is add this to the dependencies section of your Cargo.toml file:
signal-hook = "0.3.4"
While this will work for ALL of our examples, if you want to compile the complex example from the documentation, you'll want your toml file to say this instead:
signal-hook = { version = "0.3.4", features = ["extended-siginfo"] }
Simple Example
This is a slightly modified version of the simple example from the documentation:
use std::io::Error;
use std::sync::Arc;
use std::sync::atomic::{AtomicBool, Ordering};
use std::thread;
use std::time;
use signal_hook::flag;
fn main() -> Result<(), Error> {
// Declare bool, setting it to false
let term = Arc::new(AtomicBool::new(false));
// Ask signal_hook to set the term variable to true
// when the program receives a SIGTERM kill signal
flag::register(signal_hook::consts::SIGTERM, Arc::clone(&term))?;
// Do work until the term variable becomes true
while !term.load(Ordering::Relaxed) {
println!("Doing work...");
thread::sleep(time::Duration::from_secs(1));
}
// Since our loop is basically an infinite loop,
// that only ends when we receive SIGTERM, if
// we got here, it's because the loop exited after
// receiving SIGTERM
println!("Received SIGTERM kill signal. Exiting...");
Ok(())
}
Lets have a look at the different parts. First, we have this:
let term = Arc::new(AtomicBool::new(false));
This line creates a special boolean variable that can be used safely from different threads. It can be set in one thread, by the signal_hook crate, and checked in your main thread.
Next, we have this line:
flag::register(signal_hook::consts::SIGTERM, Arc::clone(&term))?;
This tells signal_hook that we would like to register the SIGTERM kill signal to be monitored, and when we receive this kill signal, to have the term boolean set to true.
Lastly, we have the main loop:
while !term.load(Ordering::Relaxed) { ... }
The way we check if the special boolean we are using is true or false is slightly different than how we check regular booleans. We use the .load() method on it. Because the loop starts out false, and we are waiting for it to turn true, and while loops only run while something is true, we use ! here. All we are saying is that we want the loop to run until the boolean turns true.
To test this program, compile it, run it, and after a few seconds, run:
killall -SIGTERM my_program
where my_program is the name of your program. That's the name you gave it when you ran crate new my_program. It's also the name of your binary under target/debug/ of your project. This will send your program a SIGTERM signal, causing the boolean to get set to true, and the loop to exit. Simple.
Double SIGINT
This is a slightly more complex example. Here, if the user hits Ctrl+C (sending our program the SIGINT kill signal), our program will begin to clean up and try to exit. If the user hits Ctrl+C again, possibly because our cleanup procedure locked up, our program will terminate immediately.
use std::io::Error;
use std::sync::Arc;
use std::sync::atomic::{AtomicBool, Ordering};
use std::thread;
use std::time;
// This is just a collection of ints that represent kill signals.
// More specifically, they are the common kill signals used to
// terminate a program
// You can do println!("{:?}", TERM_SIGNALS) to see them
// They are just SIGINT(2), SIGTERM(15) and SIGQUIT(3)
use signal_hook::consts::TERM_SIGNALS;
// Module that sets boolean flags when kill signal is received
use signal_hook::flag;
fn main() -> Result<(), Error> {
// A special boolean that can be used across threads
// It will be passed to flag::register, which will
// set it to true the first time a kill signal is received
let term_now = Arc::new(AtomicBool::new(false));
// Register all kill signals
// Note: You COULD specify other, specific kill signals here
// rather than the 3 in TERM_SIGNALS. You just need a vector
// of constants from signal_hook::consts::signal
for sig in TERM_SIGNALS {
// When terminated by a second term signal, exit with exit code 1.
// This will do nothing the first time (because term_now is false).
flag::register_conditional_shutdown(*sig, 1, Arc::clone(&term_now))?;
// But this will "arm" the above for the second time, by setting it to true.
// The order of registering these is important, if you put this one first, it will
// first arm and then terminate ‒ all in the first round.
flag::register(*sig, Arc::clone(&term_now))?;
}
// Main process that does work until term_now has been set
// to true by flag::register
while !term_now.load(Ordering::Relaxed) {
println!("Doing work...");
thread::sleep(time::Duration::from_secs(1));
}
// If we ended up here, the loop above exited because of a kill signal
println!("\nReceived kill signal. Wait 10 seconds, or hit Ctrl+C again to exit immediately.");
// This simulates a long cleanup operation
// If you wait this long, the program will exit
// If you hit Ctrl+C again before this is done, flag::register_conditional_shutdown will kill
// the process without waiting for it to finish. This means double Ctrl+C kills the process
// immediately
thread::sleep(time::Duration::from_secs(10));
println!("Exited cleanly");
Ok(())
}
This code is actually very similar to the first example, but with one additional part. Now, the term_now bool is used for 2 different purposes.
The first is defined here:
for sig in TERM_SIGNALS {
flag::register_conditional_shutdown(*sig, 1, Arc::clone(&term_now))?;
flag::register(*sig, Arc::clone(&term_now))?;
}
As before, the register function will set term_now to true when it receives a kill signal. The new register_conditional_shutdown on the other hand will kill your program, but only if term_now is set to true.
The first time you hit Ctrl+C, register_conditional_shutdown will do nothing because term_now will be false, but register will set term_now to true. The second time you hit Ctrl+C, register_conditional_shutdown will terminate the program no matter what else the program might be working on.
These 2 lines in a for loop will make the double-Ctrl+C work (terminate your program), but you still need your main code to watch for, and react to a kill signal to shut down your program gracefully. As before, we do this with a while loop that runs until term_now becomes true.
Complex Example
Building on the previous examples, the following is a slightly more complex example that reacts differently to different kill signals:
use std::io::Error;
use std::sync::Arc;
use std::sync::atomic::AtomicBool;
use std::thread;
use std::time;
use std::sync::atomic::Ordering;
use signal_hook::flag;
use signal_hook::consts::signal::*;
use signal_hook::consts::TERM_SIGNALS;
use signal_hook::iterator::Signals;
fn main() -> Result<(), Error> {
// A special boolean that can be used across threads
// The first time a kill signal is received, it will be set to
// true by flag::register
// The second time a kill signal is received, our process will
// be killed by flag::register_conditional_shutdown
let term_now = Arc::new(AtomicBool::new(false));
for sig in TERM_SIGNALS {
// When terminated by a second term signal, exit with exit code 1.
// This will do nothing the first time (because term_now is false).
flag::register_conditional_shutdown(*sig, 1, Arc::clone(&term_now))?;
// But this will "arm" the above for the second time, by setting it to true.
// The order of registering these is important, if you put this one first, it will
// first arm and then terminate ‒ all in the first round.
flag::register(*sig, Arc::clone(&term_now))?;
}
// Our actual work thread
let t = thread::spawn(move || {
while !term_now.load(Ordering::Relaxed)
{
println!("Doing work...");
thread::sleep(time::Duration::from_secs(1));
}
println!("\nThread exiting...");
});
// Create iterator over signals
let mut signals = Signals::new(TERM_SIGNALS)?;
// This loop runs forever, and blocks until a kill signal is received
'outer: loop {
for signal in signals.pending() {
match signal {
SIGINT => {
println!("\nGot SIGINT");
break 'outer;
},
SIGTERM => {
println!("\nGot SIGTERM");
break 'outer;
},
term_sig => {
println!("\nGot {:?}", term_sig);
break 'outer;
},
}
}
}
// Wait for thread to exit
t.join().unwrap();
// Cleanup code goes here
println!("\nReceived kill signal. Wait 10 seconds, or hit Ctrl+C again to exit immediately.");
thread::sleep(time::Duration::from_secs(10));
println!("Exited cleanly");
Ok(())
}
Here, a few things are going on that we are already familiar with. Just like before, we create a new bool that signal_hook will set to true for us when it receives a kill signal.
We also, once again, use flag::register_conditional_shutdown to make sure that a second SIGINT will kill our program immediately.
Something that's new here is this:
// Our actual work thread
let t = thread::spawn(move || {
while !term_now.load(Ordering::Relaxed)
{
println!("Doing work...");
thread::sleep(time::Duration::from_secs(1));
}
println!("\nThread exiting...");
});
Here, we are spawning a new worker thread where our actual work will be done, whatever that happens to be, and moving our term_now variable to this thread, so that the worker thread can monitor the boolean, and know when it's time to exit (when we've received a kill signal).
The worker thread looks very similar to our previous examples' main thread - it runs an infinite while loop that only exits if the boolean signal_hook will set for us becomes true. When it does exit, the statements following the while loop will be executed on its way out.
The next new code in this example is this:
let mut signals = Signals::new(TERM_SIGNALS)?;
This will let us use an iterator to iterate over any incoming signals.
The main example in the signal_hook documentation uses SignalsInfo instead. The main difference is that Signals will simply return the kill signal received, whereas SignalsInfo returns some additional info about every kill signal.
As stated above, using Signals doesn't require anything special, while using SignalsInfo requires the extended-siginfo feature to be enabled in your Cargo.toml, so please be aware.
After that, we have this code:
'outer: loop {
for signal in signals.pending() {
match signal {
SIGINT => {
println!("\nGot SIGINT");
break 'outer;
},
SIGTERM => {
println!("\nGot SIGTERM");
break 'outer;
},
term_sig => {
println!("\nGot {:?}", term_sig);
break 'outer;
},
}
}
}
This is a loop within a loop. The inner loop iterates over any kill signals in queue, while the outer loop is an infinite loop that keeps checking the queue.
When we do receive a kill signal, we can match it against the constants found in signal_hook::consts::signal, like SIGINT and SIGTERM, take action, and break out of both loops.
In order to easily break out of both loops, we give the outer loop a name, and our break statement tells rust that we want to break out of the loop with the given name, rather than the first enclosing loop.
After we are out of the loop, we have this line:
t.join().unwrap();
Because the only way to get out of the loop is by receiving a kill signal, and our worker thread will also see this kill signal, we can assume at this point that our thread is terminating, so here, we simply wait for it to return (finish its cleanup).
Once the thread returns, we can do our own cleanup here:
println!("\nReceived kill signal. Wait 10 seconds, or hit Ctrl+C again to exit immediately.");
thread::sleep(time::Duration::from_secs(10));
println!("Exited cleanly");
This can be whatever you need to do before your program exits.
Because we used flag::register_conditional_shutdown at the beginning, if this cleanup gets stuck, the user has the option of hitting Ctrl+C again in order to terminate our program.
Conclusion
Now that we've slowly worked up from a basic example, to a complex example, hopefully you are ready to read, and understand the example in the signal_hook documentation that uses SignalsInfo, and a few extra features we haven't covered here.
It took me a few days to figure this out - I hope with this guide, you'll be able to get going faster.
If you have any questions, please let me know. I'm still learning Rust, and definitely don't know everything about the signal_hook crate, but I'll try to answer them as best as I can.
Top comments (2)
Thanks Tal. When I first started looking for how to catch signals, I was expecting a solution similar to what I find with Python. This article did help me over the top. Or maybe down the signal hole.
Fantastic! Works for me. Searching for some tutorials to handle the kill signal, I finally solve my problem with this blog.