Roger Torres (he/him/ele)

Posted on Jul 19, 2021

Smart Pointers in Rust: What, why and how?

#beginners #rust #programming #learning

TL;DR: I will cover some of Rust's smart pointers: Box, Cell, RefCell, Rc, Arc, RwLock and Mutex.

It seems pretty obvious that smart pointers are pointers that are... smart. But what exactly does this "smart" means? When should we use them? How do they work?

These are the questions I will begin to answer here. And that is it: the beginning of an answer, nothing more. I expect that this article will give you a "background of understanding" (something like "familiarity" with the concept) that will help you accommodate a real understanding of the topic, which will come from reading the official documentation and, of course, practice.

If you are already familiar with it, you may use this text as a list of relevant reading. Look for the "useful links" at the beginning of each section.

Index:

Box
Cell
RefCell
Rc
Arc
RwLock
Mutex

Smart points in general

As explained in The Book, pointers are variables containing an address that "points at" some other data. The usual pointer in Rust is the reference (&). Smart pointers are pointers that "have additional metadata and capabilities", e.g., they may count how many times the value was borrowed, provide methods to manage read and write locks, etc.

Technically speaking, String and Vec are also smart pointers, but I will not cover them here as they are quite common and are usually thought of as types rather than pointers.

Also note that, from this list, only Arc, RwLock, and Mutex are thread-safe.

Box<T>

Useful links: The Book; Documentation; Box, stack and heap.

What?

Box<T> allows you to store T on the heap. So, if you have, say, a u64 that would be stored on the stack, Box<u64> will store it on the heap.

If you are not comfortable with the concepts of stack and heap, read this.

Why?

Values stored on the stack cannot grow, as Rust needs to know its size at compile time. The best example of how this may affect your programming that I know is in The Book: recursion. Consider the code below (and its comments).

// This does not compile. The List contains itself,  
// being recursive and therefore having an infinite size.
enum List { 
   Cons(i32, List),
   Nil,
}
// This does compile because the size of a pointer
// does not change according to the size of the pointed value.
enum List { 
   Cons(i32, Box<List>),
   Nil,
}

Be sure to read this section on The Book's to understand the details.

On a more general note, Box is useful when your value is too big to be kept on the stack or when you need to own it.

How?

To get the value T inside Box<T> you just have to deref it.

let boxed = Box::new(11);
assert_eq!(*boxed, 11)

Cell<T>

Useful links: Module documentation; Pointer documentation.

What?

Cell<T> gives a shared reference to T while allowing you to change T. This is one of the "shareable mutable containers" provided by the module std::cell.

Why?

In Rust, shared references are immutable. This guarantees that when you access the inner value you are not getting something different than expected, as well as assure that you are not trying to access the value after it was freed (which is a big chunk of those 70% of security bugs that are memory safety issues).

How?

What Cell<T> does is provide functions that control our access to T. You can find them all here, but for our explanation we just need two: get() and set().

Basically, Cell<T> allows you to freely change T with T.set() because when you use T.get(), you retrieve Copy of T, not a reference. That way, even if you change T, the copied value you got with get() will remain the same, and if you destroy T, no pointer will dangle.

One last note is that T has to implement Copy as well.

use std::cell::Cell;
let container = Cell::new(11);
let eleven = container.get();
{
    container.set(12);
}
let twelve = container.get();
assert_eq!(eleven, 11);
assert_eq!(twelve, 12);

RefCell<T>

Useful links: The Book; Documentation.

What?

RefCell<T> also gives shared reference to T, but while Cell is statically checked (Rust checks it at compile time), RefCell<T> is dynamically checked (Rust checks it at run time).

Why?

Because Cell operates with copies, you should restrict yourself to using small values with it, which means that you need references once again, which leads us back to the problem that Cell solved.

The way RefCell deals with it is by keeping track of who is reading and who writing T. That's why RefCell<T> is dynamically checked: because you are going to code this check. But fear not, Rust will still make sure you don't mess up at compile time.

How?

RefCell<T> has methods that borrow either a mutable or immutable reference to T; methods that will not allow you to do so if this action would be unsafe. As with Cell, there are several methods in RefCell, but these two are enough to illustrate the concept: borrow(), which gets an immutable reference; and borrow_mut(), which gets a mutable reference. The logic used by RefCell goes something like this:

If there is no reference (either mutable or immutable) to T, you may get either a mutable or immutable reference to it;
If there is already a mutable reference to T, you may get nothing and got to wait until this reference is dropped;
If there are one or more immutable references to T, you may get an immutable reference to it.

As you can see, there is no way to get both mutable and immutable references to T at the same time.

Remember: this is not thread-safe. When I say "no way", I am talking about a single thread.

Another way to think about is:

Immutable references are shared references;
Mutable references are exclusive references.

It is worth to say that the functions mentioned above have variants that do not panic, but returns Result instead: try_borrow() and try_borrow_mut();

use std::cell::RefCell;
let container = RefCell::new(11);
{
    let _c = container.borrow();
    // You may borrow as immutable as many times as you want,...
    assert!(container.try_borrow().is_ok());
    // ...but cannot borrow as mutable because 
    // it is already borrowed as immutable.
    assert!(container.try_borrow_mut().is_err());
} 
// After the first borrow as mutable...
let _c = container.borrow_mut();
// ...you cannot borrow in any way.
assert!(container.try_borrow().is_err());
assert!(container.try_borrow_mut().is_err());

Rc<T>

Useful links: The Book; Module documentation; Pointer documentation; Rust by example.

What?

I will quote the documentation on this one:

The type Rc<T> provides shared ownership of a value of type T, allocated on the heap. Invoking clone on Rc produces a new pointer to the same allocation on the heap. When the last Rc pointer to a given allocation is destroyed, the value stored in that allocation (often referred to as “inner value”) is also dropped.

So, like a Box<T>, Rc<T> allocates T on the heap. The difference is that cloning Box<T> will give you another T inside another Box while cloning Rc<T> gives you another Rc to the same T.

Another important remark is that we don't have interior mutability in Rc as we had in Cell or RefCell.

Why?

You want to have shared access to some value (without making copies of it), but you want to let it go once it is no longer used, i.e., when there is no reference to it.

As there is no interior mutability in Rc, it is common to use it alongside Cell or RefCell, for example, Rc<Cell<T>>.

How?

With Rc<T>, you are using the clone() method. Behind the scene, it will count the number of references you have and, when it goes to zero, it drops T.

use std::rc::Rc;
let mut c = Rc::new(11);

{ 
    // After borrwing as immutable...
    let _first = c.clone();

    // ...you can no longer borrow as mutable,...
    assert_eq!(Rc::get_mut(&mut c), None);

    // ...but can still borrow as immutable.
    let _second = c.clone();

    // Here we have 3 pointer ("c", "_first" and "_second").
    assert_eq!(Rc::strong_count(&c), 3);
}

// After we drop the last two, we remain only with "c" itself.
assert_eq!(Rc::strong_count(&c), 1);

// And now we can borrow it as mutable.
let z = Rc::get_mut(&mut c).unwrap();
*z += 1;
assert_eq!(*c, 12);

Arc<T>

Useful links: Documentation; Rust by example.

What?

Arc is the thread-safe version of Rc, as its counter is managed through atomic operations.

Why?

I think the reason why you would use Arc instead of Rc is clear (thread-safety), so the pertinent question becomes: why not just use Arc every time? The answer is that these extra controls provided by Arc come with an overhead cost.

How?

Just like Rc, with Arc<T> you will be using clone() to get a pointer to the same value T, which will be destroyed once the last pointer is dropped.

use std::sync::Arc;
use std::thread;

let val = Arc::new(0);
for i in 0..10 {
    let val = Arc::clone(&val);

    // You could not do this with "Rc"
    thread::spawn(move || {
        println!(
            "Value: {:?} / Active pointers: {}", 
            *val+i, 
            Arc::strong_count(&val)
        );
    });
}

RwLock<T>

Useful link: Documentation.

RwLock is also provided by the parking_lot crate.

What?

As a reader-writer lock, RwLock<T> will only give access to T once you are holding one of the locks: read or write, which are given following these rules:

To read: If you want a lock to read, you may get it as long as no writer is holding the lock; otherwise, you have to wait until it is dropped;
To write: If you want a lock to write, you may get as long as no one, reader or writer, is holding the lock; otherwise, you have to wait until they are dropped;

Why?

RwLock allows you to read and write the same data from multiple threads. Different from Mutex (see below), it distinguishes the kind of lock, so you may have several read locks as far as you do not have any write lock.

How?

When you want to read a RwLock, you got to use the function read()—or try_read()—that will return a LockResult that contains a RwLockReadGuard. If it is successful, you will be able to access the value inside RwLockReadGuard by using deref. If a writer is holding the lock, the thread will be blocked until it can take hold of the lock.

Something similar happens when you try to use write()—or try_write(). The difference is that it will not only wait for a writer holding the lock, but for any reader holding the lock as well.

use std::sync::RwLock;
let lock = RwLock::new(11);

{
    let _r1 = lock.read().unwrap();
    // You may pile as many read locks as you want.
    assert!(lock.try_read().is_ok());
    // But you cannot write.
    assert!(lock.try_write().is_err());
    // Note that if you use "write()" instead of "try_write()"
    // it will wait until all the other locks are released
    // (in this case, never).
} 

// If you grab the write lock, you may easily change it
let mut l = lock.write().unwrap();
*l += 1;
assert_eq!(*l, 12);

If some thread holding the lock panics, further attempts to get the lock will return a PoisonError, which means that from then on every attempt to read RwLock will return the same PoisonError. You may recover from a poisoned RwLock using into_inner().

use std::sync::{Arc, RwLock};
use std::thread;

let lock = Arc::new(RwLock::new(11));
let c_lock = Arc::clone(&lock);

let _ = thread::spawn(move || {
    let _lock = c_lock.write().unwrap();
    panic!(); // the lock gets poisoned
}).join();

let read = match lock.read(){
    Ok(l) => *l,
    Err(poisoned) => {
        let r = poisoned.into_inner();
        *r + 1
    }
};

// It will be 12 because it was recovered from the poisoned lock
assert_eq!(read,12);

Mutex<T>

Useful links: The Book; Documentation.

Mutex is also provided by the parking_lot crate.

What?

Mutex is similar to RwLock, but it only allows one lock-holder, no matter if it is a reader or a writer.

Why?

One reason to prefer Mutex over RwLock is that RwLock may lead to writer starvation (when the readers pile on and the writer never gets a chance to take the lock, waiting forever), something the does not happen with Mutex.

Of course, we are diving into deeper seas here, so the real-life choice falls on more advanced considerations, such as how many readers you expect at the same time, how the operating system implements the locks, and so on...

How?

Mutex and RwLock work in a similar fashion, the difference is that, because Mutex does not differentiate between readers and writers, you just use lock() or try_lock to get the MutexGuard. The poisoning logic also happens here.

use std::sync::Mutex;
let guard = Mutex::new(11);

let mut lock = guard.lock().unwrap();
// It does not matter if you are locking the Mutex to read or write,
// you can only lock it once.
assert!(guard.try_lock().is_err());

// You may change it just like you did with RwLock
*lock += 1;
assert_eq!(*lock, 12);