DEV Community: Grayson Hay

Rust Snippet: Exec with Update Intervals

Grayson Hay — Thu, 30 May 2024 15:55:40 +0000

Recently, I needed to kick off a long running process for encoding videos using ffmpeg in the middle of handling an SQS message. In order to keep the message from showing back up in the queue before the video processing is finished.

So that means, I need to be able to send the change message visibility timeout periodically during the process. so, I came up with this little function to help. It calls a “progress” function every 10 seconds while the command is executing, and then ends once it’s done.

Using tokio as the Async Runtime

pub async fn exec<F, Fut>(mut cmd: Command, progress: F) -> Result<Output, ProgressingExecError>
where
    F: Fn() -> Fut,
    Fut: Future<Output = ()>,
{
    let (tx, mut rx) = oneshot::channel();
    let mut interval = interval(Duration::from_millis(10000));

    tokio::spawn(async move {
        let output = cmd.output().await;
        let _ = tx.send(output);
    });

    loop {
        tokio::select! {
            _ = interval.tick() => progress().await,
            msg = &mut rx => return Ok(msg??)
        }
    }
}

Opaque Types for UniFFI

Grayson Hay — Thu, 07 Mar 2024 06:44:41 +0000

I'm going to show you how to use slab to send handles from rust to python and back to rust using an arena allocator called slab.

tl;dr

With slab, we allocate types into a chunk of memory that has been pre-allocated and get an id back. It's really just an index into the chunk of memory, indexed by the size of the type we are using.

We have to have a "global" (static in some way) chuck of this memory to handle the FFI nature of it.

The example code from this article is available here: https://github.com/graysonarts/uniffi-slab-example

Background

On my youtube series "Growing up Rust", I'm building a personal CRM in Rust with a Swift frontend. I'm using CQRS and an event-driven architecture with the least amount of swift as possible. I'm using UniFFI to generate the bindings for swift (and in this example python)

To accomplish this, I need the ability to create commands and then submit the command to the core rust model for processing.

I'm handling this part of it by an enum called PersonCommand which handles all the different types of changes that can happen to a person. I want to treat the PersonCommand as completely opaque to the swift code. It just needs to know how to create the necessary commands and submit them to the run queue.

Because I want to batch up commands at the dialog boundary, I need swift to be able to hold onto the Command before it's processed, but it doesn't need access to anything inside of the command, so I want it to be opaque. UniFFI doesn't support this out of the box (that I could find, let me know if I'm wrong!).

The key pieces.

I'm not going to cover setting up UniFFI so make sure you check their documentation if you need help there.

The global slab

In order for slab to be global, we need to make it a static variable. There are other patterns you could use for this, but this is the simplest.

static COMMAND_SLAB: Lazy<Mutex<Slab<OpaqueType>>>
   = Lazy::new(|| Mutex::new(Slab::new()));

Lazy is from the once_cell crate. I tried to use the std::cell::LazyCell but I couldn't get it working and is not a standard feature yet, so I'd rather bring in once_cell which does work well.

We need the Mutex because the slab may be accessed from multiple threads. Other kinds of locks would work like a RwLock but again, Mutex is simple and straightforward. Your mileage may vary depending on your use case and workload. In a UI application, all commands are generated as a result of user actions, so we should be okay.

The handle type alias

I decided to use a type alias for code clarity, but it's just a u32. Technically, slab returns a usize but that can't be marshalled across the FFI, so we cast it to a u32. Adjust as necessary. It might be safer to serialize and deserialize to a string using serde or some other crate.

type OpaqueHandle = u32;

The factory method

Because it's an opaque type to our FFI, we need to have a function that returns the handle for specific instance of the type.

fn ffi_make_opaque(number: u32) -> OpaqueHandle {
    COMMAND_SLAB.lock().unwrap().insert(OpaqueType {
        ...
    }) as OpaqueHandle
}

Accepting the handle

Now if we need to send the handle back into rust, we would do something like this

    let slab = COMMAND_SLAB.lock().unwrap();
    let command = slab.get(handle as usize).unwrap();
    println!("Processing handle: {:?}", command));

In this case, we need to bind the slab to a local variable to control the lifetime, and then we get the command from the slab and do something with it.

Ideally, this function would get the command and then call it's execute function.

Alternative Approaches

If we wanted, we could get the raw pointer to the type and pass that around as a value between the different languages, but I found that it was a messier and more error prone way to handle it than to use slab.

Dropping root privileges in rust

Grayson Hay — Tue, 08 Jan 2019 16:33:45 +0000

When I was writing a fingerd daemon in Rust (why? because I could), one thing that took me a little while to figure out was how to drop root privileges after I bound to port 79.

One of the principles of security is to use the least amount of privileges you need in order to accomplish the functionality you need. In the case of fingerd and most network daemons that run on privileged ports (i.e. ports < 1024), you need to be root long enough to bind the socket to the port, and then you have no need for root. It's called the principle of least privilege.

In the world of C and C++, you have the standard C library which provides setuid(2) to change the uid of the current running process, so that's what we need to use. But from the usability standpoint, I wanted users to be able to specify the name of the user to run as, and setuid(2) takes a uid not a username, so we also need to use getpwnam(3) to use the password entry for the user by name.

To accomplish this, I created two functions, and then combined them together to create a third that drops from root to a named user. They are pretty straight forward once you figure out all of the ffi magic. This blog post will hopefully help you avoid having to worry about the ffi magic. I can't decide if I want to turn this into a crate or not, but maybe leave a comment to let me know.

First, let's get the uid from the username.

use std::io::Result;

fn get_uid(username : &str) -> Result<uid_t> {
    let p : &passwd = unsafe {
        let cstr = CString::new(username).expect("Unable to pass username to underlying C library");
        let p = getpwnam(cstr.as_ptr());
        if p.is_null() {
            return Err(Error::from_raw_os_error(ENOENT));
        }
        &*p
    };

    Ok(p.pw_uid)
}

So, there is some unsafe code to call getpwname(3). We convert the &str username into a CString so we can pass it into getpwnam(3). Then we call the system call, check to make sure it's not null, and then return it as a reference outside of the unsafe block. Once we have it, then we grab the pw_uid which is guaranteed to contain a value if we returned anything.

fn shed(uid : uid_t) -> Result<()> {
    match unsafe { setuid(uid) } {
        0 => Ok(()),
        -1 => Err(Error::last_os_error()),
        n => unreachable!("setuid returned {}", n)
    }
}

Next up, we wrap setuid(2) in an unsafe and convert the integer return code into a std::io::Result.

pub fn drop_root(username : &str) -> Result<()> {
    get_uid(username).and_then(shed)
}

Lastly, we chain them together via the .and_then combinator, and then to use it, we do drop_root("rhay")?. I do this immediately after TcpListener::bind(&addr).unwrap(); in my main function. Since main() can now return a Result, ? works great.

I hope this post saved you a little time. When I was working on this, I couldn't find any reference that explicitly called out this common security pattern in Rust.

Bitflags 1.0.1 - Part 1

Grayson Hay — Tue, 21 Nov 2017 21:00:00 +0000

this is a post from my blog that I wrote a while ago, I thought I had posted it to dev.to, but apparently, I didn't. Bitflags is currently at 1.2.1 so this is a little out of date with the current RustLang

This is an attempt for me to get a better understanding of Rust by looking at real work examples and explaining what they do. I am far from a competent rust programmer, so if I get something wrong, please leave a comment with the corrections and I’ll update the posts based on your feedback with appropriate edit credits.

The first in the series will be the bitflags module. I'm linking to the 1.0.1 version since that was the latest version when I started writing this post. I'll link to important lines, but will also include some lines directly from the Crate to talk about it.

What does the Crate do?

The bitflags crate creates C-Style bitmask fields. You know, those things where you OR values together to configure "things". You see this alot in old C APIs, and OS level APIs. This crate allows you to define those in a type-safe manner and pass them to C APIs.

The Entry Macro

The entry point into the crate is the bitflags! macro. This is where the magic happens.

#[macro_export]
macro_rules! bitflags {

We start off with the macro rule for the macro. Nothing special here, this is the standard way to define a macro in Rust, but macros in Rust are a bit weird, so the next parts take a bit of digging for me to understand. A little peek into the Rust documentation on macros gave me a bit to work
with.

    (
        $(#[$outer:meta])*
        pub struct $BitFlags:ident: $T:ty {    
            $(
                $(#[$inner:ident $($args:tt)*])*
                const $Flag:ident = $value:expr;
            )+
        }

This first bit is for defining the contents inside of the macro, but it's a bit obtuse if you don't know what you are looking at, so let's break this down even further.

$(#[$outer:meta])*

If you recall how the bitflags! macro is used, it contains a struct. This bit means that you can attach outside attributes (ones in the form of #[...]) to the struct. In this case, we are naming them $outer for later use in the generated code.

pub struct $BitFlags:ident: $T:ty {

Next in the macro is the ability to have a struct defined inside of the invocation body. The $BitFlags:ident creates a variable and it holds an identifier, in this case, whatever you named your struct when you called the macro. The $T:ty part is saying that you also need to specify a type for the bit flags that will be generated. Even though this isn't encoded in the macro, it should be an unsigned integer type of some sort.

The rest of the above block is similar to the first bits, and it defines the ability to specify lines like const A = 0b00000001; within the struct.

    ) => {
        __bitflags! {

The end of the pattern matching is the beginning of what is going to be generated from this macro. The code invokes another macro which we'll cover in a bit when we look at the implementation.

            $(#[$outer])*
            (pub) $BitFlags: $T {
                $(
                    $(#[$inner $($args)*])*
                    $Flag = $value;
                )+
            }
        }
    };

The rest of this macro is the body that will be passed into the implementation macro. It passes the attributes and any of the const declarations into body and apparently stripping the const from them.

The rest of these first macros are about the same with the main differences just covering visibility and other modifiers on the struct. I won't retread those bits, so let's move on to the implementation details.

The Inner Workings

#[macro_export]
#[doc(hidden)]
macro_rules! __bitflags {
    (
        $(#[$outer:meta])*
        ($($vis:tt)*) $BitFlags:ident: $T:ty {
            $(
                $(#[$inner:ident $($args:tt)*])*
                $Flag:ident = $value:expr;
            )+
        }

Here, we are defining another pattern for the macro. This code is all very similar to the previous pattern except that we now include the visiblility in the pattern. I think the main reason they chose to repeat the macro pattern for define visibility in the public interface was to be explicit about which types of visibility are supported.

    ) => {
        #[derive(Copy, PartialEq, Eq, Clone, PartialOrd, Ord, Hash)]
        $(#[$outer])*
        $($vis)* struct $BitFlags {
            bits: $T,
        }

Now we start generating some real rust code. This defines a struct that will be named $BitFlags with a single member called bits. It also derives implementations for a bunch of useful traits that will make working with the bit flags easier.


        __impl_bitflags! {
            $BitFlags: $T {
                $(
                    $(#[$inner $($args)*])*
                    $Flag = $value;
                )+
            }
        }
    };
}

And were we have another indirection into another implementation specific macro. The __impl_bitflags is a pretty big macro, so I'm breaking that out into a separate article. Part 2 coming soon

If you enjoyed this post, or if you have any feedback, corrections, or anything, please leave a comment below.