Deconstructing SPF Records with Rust

Hi. If you have visited my blog before you might have seen my series on using the trust-dns-resolver crate. As part of that series I looked at querying DNS TXT records. During the process I was reminded of SPF records. Something I used to deal with in a previous position. And this got me to thinking about using SPF records as a way to learn more about working with rust.

Keep in mind I am new to rust and these will just be some of my thoughts as I work through the task I have given myself. This will not be a production project, merely a learning experience. I will not be documenting all the code here. I will give a reference to the git repository at the end of this article.

TL;DR;

Some caveats

• I will not be implementing a full solution to deconstruct an SPF record at this time.
• Not all edge cases may be covered, though I will do my best to do so.
• I will initially only be looking at: redirect, include, ip4 and ip6 records contained within the SPF record.
• If there is no qualifier present, it is assumed to be + which equates to a Pass
• During this phase I will not be looking at using generics as I want to understand the basics of using structs
• I will not be writing any code that validates that the SPF record is syntactically correct.
• The code is going to be fairly ugly and will get polished as I go through.
• I am not looking at SPF2 (pra/mfrom)

Some SPF rules I am fairly familiar with, though I could be mistaken. (smile)

• There can only be one redirect in a single SPF record.
• The qualifier is optional
• There can be multiple instances of ip4, ip6 and include
• There can only be a single a and mx when they are not followed by a : character.
• With the exception of a redirect there must an all at the end of an SPF record.

Let's get started

I am not going to start off with how to create a rust project. There is plenty on that out there. And if you read my previous series on Working with the trust-dns-resolver Crate, you already know how to get started. In fact this is an extension to that series.

Structs

Following the basic rules I listed above each mechanism can have an optional qualifier. This will need to be dealt with. And each mechanism can occur multiple times in most cases.

The Include Struct

An include can be represented in the following ways: (Not an exhaustive list)

• include:_spf.example.com
• +include:_spf.otherdomain.com
• -include:_spf.somedomain.com

#[derive(Default, Debug, Clone)]
struct Include {
    qualifier: char,
    txt: String,
}

Code Explanation

• I will store the qualifier as a char, this will allow me to hold +,-,~, or ?. Alternatively I could create an enum type. But this is over kill since I am just trying to learning at this point.
• I will store the _spf.example.com portion in a String which lives on the heap in memory.
• To save on some code, I am asking the compiler to generate Default, Debug and Clone for me using the #[derive]

Implementing the Include Struct

Rust is interesting in that it allows us to create functions which are tied to a struct. In my mind this is some what like the struct is acting as an object, and the functions are then used as methods of the struct.

impl Include {
    fn new(qualifier: char, txt: String) -> Self {
        Self { qualifier, txt }
    }
    fn as_mechanism(&self) -> String {
        let mut txt = String::new();
        if self.qualifier != '+' {
            txt.push(self.qualifier);
        }
        txt.push_str("include:");
        txt.push_str(self.txt.as_str());
        txt
    }
    fn is_pass(&self) -> bool {
        if self.qualifier == '+' {
            true
        } else {
            false
        }
    }
    fn is_fail(&self) -> bool {
        if self.qualifier == '-' {
            true
        } else {
            false
        }
    }
    fn is_softfail(&self) -> bool {
        if self.qualifier == '~' {
            true
        } else {
            false
        }
    }
    fn is_neutral(&self) -> bool {
        if self.qualifier == '?' {
            true
        } else {
            false
        }
    }
}

Explanation

The function new allows us to instantiate and struct with the passed values. We can call it in the following ways

let var_a = Include::new('+', "_spf.example.com".as_string());

• Single quotes can be used here to denote a type of char
• Text enclosed in double-quotes are technically of type str and since my function definition expects a String, I am transforming the string. This is purely for the example purposes. I could have also easily passed String::from("_spf.example.com")

The is_* functions all simple check the value stored in qualifier and return a boolean.

The as_mechanism() function is able to reconstruct the spf mechanism and return it as a String. If the qualifier is NOT a +, it is appended to the string being created. Otherwise it is omitted.

The Ip4 Struct

An ip4 can be presented in the following ways: (not an exhaustive list)

• ip4:157.55.9.128/25
• +ip4:ip4:72.14.192.0/18
• Or as a single host ip4:X.X.X.X which would be used as X.X.X.X/32

#[derive(Debug, Clone)]
struct Ip4 {
    qualifier: char,
    ip: IpNetwork,
}

Here I am only using Debug and Clone as I am implementing my own Default merely for learning purposes. The astute reader might already conclude that this struct is is very similar to Include with the only real difference being String and IpNetwork. IpNetwork is provided by the ipnetwork crate. I am using this as it provides lots of the basic functions I will need and also many more that would make it useful for other applications.

impl Default for Ip4 {
    fn default() -> Self {
        Self {
            qualifier: '+',
            ip: IpNetwork::V4("0.0.0.0/0".parse().unwrap()),
        }
    }
}

With his Default we can call code such as:

let ip: Ip4 = ip4::default();

Implementing the Ip4 Struct

impl Ip4 {
    fn new(qualifier: char, ip: IpNetwork) -> Self {
        Self { qualifier, ip }
    }
    fn as_string(&self) -> String {
        self.ip.to_string()
    }
    fn as_mechanism(&self) -> String {
        let mut ip4_string = String::new();
        if self.qualifier != '+' {
            ip4_string.push(self.qualifier);
        }
        ip4_string.push_str("ip4:");
        ip4_string.push_str(self.ip.to_string().as_str());
        ip4_string
    }
    fn as_ip(&self) -> IpNetwork {
        self.ip
    }
    fn is_pass(&self) -> bool {
        if self.qualifier == '+' {
            true
        } else {
            false
        }
    }
    fn is_fail(&self) -> bool {
        if self.qualifier == '-' {
            true
        } else {
            false
        }
    }
    fn is_softfail(&self) -> bool {
        if self.qualifier == '~' {
            true
        } else {
            false
        }
    }
    fn is_neutral(&self) -> bool {
        if self.qualifier == '?' {
            true
        } else {
            false
        }
    }
}

As you can see I have had to duplicate a lot of code.... I think generic is going to be in the near future.

The Ip6 Struct

To keep this a little shorter I will just give the basics since it is identical to Ip4

#[derive(Debug, Clone)]
struct Ip6 {
    qualifier: char,
    ip: IpNetwork,
}

impl Ip6 {
    fn new(qualifier: char, ip: IpNetwork) -> Self {
        Self { qualifier, ip }
    }
}

impl Default for Ip6 {
    fn default() -> Self {
        Self {
            qualifier: '+',
            ip: IpNetwork::V6("FE80::1".parse().unwrap()),
        }
    }
}

This is essentially the same as Ip4 because IpNetork is an enum of V4 and V6

Defining the SPF1 Struct

Many of the mechanisms in SPF are optional and nicely, rust actually provides the type Option. Also a mechanism can appear several times, as a list and rust provides Vec which is a way to store a list of some type.

#[derive(Default, Debug)]
struct Spf1 {
    source: String,
    include: Option<Vec<Include>>,
    redirect: Option<String>,
    is_redirected: bool,
    a: Option<Vec<A>>,
    mx: Option<Vec<Mx>>,
    ip4: Option<Vec<Ip4>>,
    ip6: Option<Vec<Ip6>>,
    all_qualifier: char,
}

Explanation of the SPF1 Struct

• source: The original string received from the TXT DNS lookup. This will be deconstructed to populate the SPF struct.
• include: This is an optional list of includes so I am creating a Vec of Include and then wrapping that inside and Option since it might actually be None
• Ip4 and Ip6 are handle in the same way as Include
• redirect: Only ever occurs once in a record. So I am placing it directly within SPF1, but again it is optional, so I wrap it inside an option
• is_redirect: Is closely tied to direct and simply stores a bool value
• all_qualifier: This is a special case. It is only ever not present when the spf record is a redirect so I am only ensure that I have a qualifer value.

Implementing SPF1 Struct

impl Spf1 {
    fn new(str: &String) -> Self {
        Self {
            source: str.clone(),
            include: None,
            redirect: None,
            is_redirected: false,
            a: None,
            mx: None,
            ip4: None,
            ip6: None,
            all_qualifier: '+',
        }
    }
    fn parse(&mut self) {
        let records = self.source.split_whitespace();
        let mut vec_of_includes: Vec<Include> = Vec::new();
        let mut vec_of_ip4: Vec<Ip4> = Vec::new();
        let mut vec_of_ip6: Vec<Ip6> = Vec::new();
        for record in records { // Main loop
            if record.contains("redirect=") {
                let items = record.rsplit("=");
                for item in items {
                    self.redirect = Some(item.to_string());
                    break;
                }
                self.is_redirected = true;
            } else if record.contains("include:") {
                let qualifier_and_modified_str = return_and_remove_qualifier(record, 'i');
                for item in record.rsplit(":") {
                    vec_of_includes.push(Include::new(
                        qualifier_and_modified_str.0,
                        item.to_string(),
                    ));
                    break; // skip the 'include:'
                }
            } else if record.contains("ip4:") {
                let qualifier_and_modified_str = return_and_remove_qualifier(record, 'i');
                if let Some(raw_ip4) = qualifier_and_modified_str.1.strip_prefix("ip4:") {
                    let network: Ip4 =
                        Ip4::new(qualifier_and_modified_str.0, raw_ip4.parse().unwrap());
                    vec_of_ip4.push(network);
                }
            } else if record.contains("ip6:") {
                let qualifier_and_modified_str = return_and_remove_qualifier(record, 'i');
                if let Some(raw_ip6) = qualifier_and_modified_str.1.strip_prefix("ip6:") {
                    let network: Ip6 =
                        Ip6::new(qualifier_and_modified_str.0, raw_ip6.parse().unwrap());
                    vec_of_ip6.push(network);
                }
            } else if record.ends_with("all") {
                self.all_qualifier = return_and_remove_qualifier(record, 'a').0;
            }
        } // End of Main Loop
        if vec_of_includes.len() > 0 {
            self.include = Some(vec_of_includes);
        };
        if vec_of_ip4.len() > 0 {
            self.ip4 = Some(vec_of_ip4);
        };
        if vec_of_ip6.len() > 0 {
            self.ip6 = Some(vec_of_ip6);
        };
    }
    // Code omitted for brevity
}

Explanation

let mut spf = Spf1::new(&some_spf_record.to_string());

This function takes in a reference, borrows, a string and then internally uses clone() to make a copy of the string which then remains valid for the life time of the struct. Is also initialises the variables within the struct.

Deconstructing time (Finally)

Enter the real work horse in this process. It's currently a little monolithic at the moment. But, it's getting the job done.

spf.parse();

• The first four lines set things up.
  • Split the string in to seperate parts based on white space. Placing them into an iter()
  • Create our mutable Vec varibles for Include, Ip4 and Ip6
• Next we start the main loop which will parse each mechanism
  • redirect:
  • If the current record contains rediect= we use rsplit() which returns a new iter() and access the right side of the split first. We wrap that result into a Some() because Spf1.redirect expects an Option. Next we break from the loop since we don't need any other parts of this iter(). We are also not concerned about the qualifier in this case as it is usually omitted.
  • include:
  • Similar to the redirect we do an rsplit() but on ":"
  • Create a new Include struct and push that value on to the vec_of_includes that was defined at the start of parse()
  • Call break to exist inner loop
  • ip6 has an issue. I am using strip_prefix() because I am unable to use rsplit() due to ip6 being comprised of multiple : characters. This is an issue because I am using strip_prefix() on "ip6:" This will fail if there is a qualifer present in the string.
  • ie: "+ip6:....". To handle this I have a helper function return_and_remove_qualifier() which checks for the presence of a qualifier, and returns a tuple. The tuple contains the qualifier and an updated version of original string with the qualifier removed if it was present. This ensure that the strip_prefix() function will match correctly.
  • ip4/6:
  • Call return_and_remove_qualifier(original_string, 'i')
  • strip_prefix() returns Some(), so we take a peek inside and conditionally create an Ip6 struct.
  • Push it on vec_of_ip6, I actually do the same process for ip4
  • all;
  • If the the mechanism ends with all we just store the qualifier directly into self.all_qualifier
  • When we reach the end of the main loop we then conditionally wrap our various vec_of.* structs into their own Some() and assign them to their matching property in Spf1

## Test Domain and Test Output.

### Test Domains
I am using the following records at the time of writing.

Domain	Spf Record
gmail.com	v=spf1 redirect=_spf.google.com
_netblocks.google.com	v=spf1 ip4:35.190.247.0/24 ip4:64.233.160.0/19 ip4:66.102.0.0/20 ip4:66.249.80.0/20 ip4:72.14.192.0/18 ip4:74.125.0.0/16 ip4:108.177.8.0/21 ip4:173.194.0.0/16 ip4:209.85.128.0/17 ip4:216.58.192.0/19 ip4:216.239.32.0/19 ~all
_netblocks2.google.com	v=spf1 ip6:2001:4860:4000::/36 ip6:2404:6800:4000::/36 ip6:2607:f8b0:4000::/36 ip6:2800:3f0:4000::/36 ip6:2a00:1450:4000::/36 ip6:2c0f:fb50:4000::/36 ~all
hotmail.com	v=spf1 ip4:157.55.9.128/25 include:spf.protection.outlook.com include:spf-a.outlook.com include:spf-b.outlook.com include:spf-a.hotmail.com include:_spf-ssg-b.microsoft.com include:_spf-ssg-c.microsoft.com ~all

Test Output (gmail.com)

List of TXT records found for gmail.com.
TXT Record 1:
globalsign-smime-dv=CDYX+XFHUw2wml6/Gb8+59BsH31KzUr6c1l2BPvqKX8=
TXT Record 2:
v=spf1 redirect=_spf.google.com

Decontructing SPF Record
SPF1: v=spf1 redirect=_spf.google.com

There are no include elements
There are no ip4 networks
There are no ip4 spf records.

Is a redirect: true

redirect: _spf.google.com
mechanism: redirect=_spf.google.com

Test Output (_netblocks.google.com)

List of TXT records found for _netblocks.google.com.
TXT Record 1:
v=spf1 ip4:35.190.247.0/24 ip4:64.233.160.0/19 ip4:66.102.0.0/20 ip4:66.249.80.0/20 ip4:72.14.192.0/
18 ip4:74.125.0.0/16 ip4:108.177.8.0/21 ip4:173.194.0.0/16 ip4:209.85.128.0/17 ip4:216.58.192.0/19 i
p4:216.239.32.0/19 ~all

Decontructing SPF Record
SPF1: v=spf1 ip4:35.190.247.0/24 ip4:64.233.160.0/19 ip4:66.102.0.0/20 ip4:66.249.80.0/20 ip4:72.14.
192.0/18 ip4:74.125.0.0/16 ip4:108.177.8.0/21 ip4:173.194.0.0/16 ip4:209.85.128.0/17 ip4:216.58.192.
0/19 ip4:216.239.32.0/19 ~all

There are no include elements
List of ip4 networks/hosts:
35.190.247.0/24
64.233.160.0/19
66.102.0.0/20
66.249.80.0/20
72.14.192.0/18
74.125.0.0/16
108.177.8.0/21
173.194.0.0/16
209.85.128.0/17
216.58.192.0/19
216.239.32.0/19

List of ip4 mechanisms:
ip4:35.190.247.0/24
ip4:64.233.160.0/19
ip4:66.102.0.0/20
ip4:66.249.80.0/20
ip4:72.14.192.0/18
ip4:74.125.0.0/16
ip4:108.177.8.0/21
ip4:173.194.0.0/16
ip4:209.85.128.0/17
ip4:216.58.192.0/19
ip4:216.239.32.0/19

Is a redirect: false

Test Output (_netblocks2.google.com) / Not being displayed at this time.

List of TXT records found for _netblocks2.google.com.
TXT Record 1:
v=spf1 ip6:2001:4860:4000::/36 ip6:2404:6800:4000::/36 ip6:2607:f8b0:4000::/36 ip6:2800:3f0:4000::/3
6 ip6:2a00:1450:4000::/36 ip6:2c0f:fb50:4000::/36 ~all

Decontructing SPF Record
SPF1: v=spf1 ip6:2001:4860:4000::/36 ip6:2404:6800:4000::/36 ip6:2607:f8b0:4000::/36 ip6:2800:3f0:40
00::/36 ip6:2a00:1450:4000::/36 ip6:2c0f:fb50:4000::/36 ~all

There are no include elements
There are no ip4 networks
There are no ip4 spf records.

Is a redirect: false

Test Output (hotmail.com)

List of TXT records found for hotmail.com.
TXT Record 1:
google-site-verification=gqFmgDKSUd3XGU_AzWWdojRHtW3_66W_PC3oFvQVZEw
TXT Record 2:
v=spf1 ip4:157.55.9.128/25 include:spf.protection.outlook.com include:spf-a.outlook.com include:spf-
b.outlook.com include:spf-a.hotmail.com include:_spf-ssg-b.microsoft.com include:_spf-ssg-c.microsof
t.com ~all

Decontructing SPF Record
SPF1: v=spf1 ip4:157.55.9.128/25 include:spf.protection.outlook.com include:spf-a.outlook.com includ
e:spf-b.outlook.com include:spf-a.hotmail.com include:_spf-ssg-b.microsoft.com include:_spf-ssg-c.mi
crosoft.com ~all

Include Mechanisms:
include:spf.protection.outlook.com
include:spf-a.outlook.com
include:spf-b.outlook.com
include:spf-a.hotmail.com
include:_spf-ssg-b.microsoft.com
include:_spf-ssg-c.microsoft.com
List of ip4 networks/hosts:
157.55.9.128/25

List of ip4 mechanisms:
ip4:157.55.9.128/25

Is a redirect: false

You can get a copy of the complete code at the time of this writing here

I think I will next work on using generics

Thanks for reading.