DEV Community: Al

Building an Archiver in Rust

Al — Tue, 19 Dec 2023 20:53:09 +0000

Note
Probably won't work on windows, not tested on windows and because of how it sets up paths with the forward slashes, it may not work on windows.

tldr: code

Prereqs: having Rust installed

This is a tutorial for building a basic archiver with Reqwest. If you want to try to do js heavy sites, i suggest you use Fantoccini (jonhoo is a beast). Granted a lot of sites don't like bots so they take a lot of steps with antibots. Though I've heard people doing some good things with Selenium. If you go this route you'd want to change things like user agents, options and some other stuff. In this code we'll only be doing user agent changing within reqwest.

Note
the archive uses the path and builds directories on it. So
rustlang.org/rust/language
will translate to
rustlang.org/rust/language/date/index.html it will also have an image folder, a css folder and a js folder.

So lets start setting up the project.

run

cargo new <Project Name>

rename to what ever you want to name it.

Now in the Cargo.toml file, add the following dependencies:

[dependencies]
url = "2.5.0"
reqwest = "0.11.23"
bytes = "1.5"
dirs = "5.0.1"
regex = "1.10.2"
scraper = "0.18.1"
lazy_static = "1.4.0"
chrono = "0.4.31"
rand = "0.8.5"
image = "0.24.6"
tokio = { version = "1.35", features = ["full"] }

Lets add the ways to parse necessary links for things like img's, css, js. Things pertinent to a webpage.
Create a file under ./src called html.rs

Now lets add the required dependencies:

use chrono::Utc;
use lazy_static::lazy_static;
use regex::Regex;
use scraper::{Html, Selector};
use std::collections::HashSet;
use url::{ParseError, Url};

Now those are out of the way, time to add add the HtmlRecord struct.

#[derive(Debug)]
pub struct HtmlRecord {
    pub origin: String,
    pub date_time: String,
    pub body: String,
    pub html: Html,
}

Now lets implement the new:

impl HtmlRecord {
    pub fn new(origin: String, body: String) -> HtmlRecord {
        HtmlRecord {
            origin,
            date_time: Utc::now().format("%d-%m-%Y-%H:%M:%S").to_string(),
            html: Html::parse_document(&body),
            body,
        }
    }
}

This basically takes the time of making and attaches it to the date_time field. The html field is the html parsed from the body which we will use to get links and stuff. The body is the html we will save.

Next, under new add the following method:


    //the tuple returns the unparsed string in the 0's spot
    //returns the parsed link in the 1's spot
    pub fn get_image_links(&self) -> Result<Option<HashSet<(String, String)>>, String> {
        //checks for base64 images
        lazy_static! {
            static ref RE3: Regex = Regex::new(r";base64,").unwrap();
        }
        let mut link_hashset: HashSet<(String, String)> = HashSet::new();

        //select image tags
        let selector = Selector::parse("img").unwrap();

        //loop through img tags
        for element in self.html.select(&selector) {
            //grab the source attribute of the tag
            match element.value().attr("src") {
                //if we have a link
                Some(link) => {
                    //see if a relative link
                    if Url::parse(link) == Err(ParseError::RelativeUrlWithoutBase) {
                        //get base url
                        let plink = Url::parse(&self.origin)
                            .expect("get css links, origin could not be parsed")
                            .join(link)
                            .expect("css links, could not join")
                            .to_string();
                        //push to return vector
                        link_hashset.insert((link.to_string(), plink.to_string()));
                        //check if base64 and continue if so
                    } else if RE3.is_match(link) {
                        continue;
                    //if fully formed link, push to return vector
                    } else if let Ok(parsed_link) = Url::parse(link) {
                        link_hashset.insert((link.to_string(), parsed_link.to_string()));
                    }
                }
                //No src, contine
                None => continue,
            };
        }
        //If hashset is empty return an Ok of None
        if link_hashset.is_empty() {
            Ok(None)
        //return some image links
        } else {
            Ok(Some(link_hashset))
        }
    }

I commented the code, if you have any questions feel free to ask.

Next we want to add the get_css_links under get_image_links

    pub fn get_css_links(&self) -> Result<Option<HashSet<(String, String)>>, String> {
        let mut link_hashset: HashSet<(String, String)> = HashSet::new();

        //get links
        let selector = Selector::parse("link").unwrap();
        //loop through elements
        for element in self.html.select(&selector) {
            //check if stylesheets
            if element.value().attr("rel").unwrap() == "stylesheet" {
                //get the href
                match element.value().attr("href") {
                    Some(link) => {
                        //take care of relative links here
                        if Url::parse(link) == Err(ParseError::RelativeUrlWithoutBase) {
                            //create url
                            let plink = Url::parse(&self.origin)
                                .expect("get css links, origin could not be parsed")
                                .join(link)
                                .expect("css links, could not join")
                                .to_string();
                            //add to hashset
                            link_hashset.insert((link.to_string(), plink.to_string()));
                        } else if let Ok(parsed_link) = Url::parse(link) {
                            link_hashset.insert((link.to_string(), parsed_link.to_string()));
                        }
                    }
                    None => continue,
                };
            }
        }

        if link_hashset.is_empty() {
            Ok(None)
        } else {
            Ok(Some(link_hashset))
        }
    }

Now lets lets add the get_js_links under the get_css_links

//get js links
    pub fn get_js_links(&self) -> Result<Option<HashSet<(String, String)>>, String> {
        //create hashset
        let mut link_hashset: HashSet<(String, String)> = HashSet::new();
        //get the selector which is basically used for getting the script tags
        let selector = Selector::parse("script").unwrap();
        for element in self.html.select(&selector) {
            //get src attribute of the script tag
            match element.value().attr("src") {
                Some(link) => {
                    if Url::parse(link) == Err(ParseError::RelativeUrlWithoutBase) {
                        //parse relative url
                        let plink = Url::parse(&self.origin)
                            .expect("get js links, origin could not be parsed ")
                            .join(link)
                            .expect("js links, could not join")
                            .to_string();
                        link_hashset.insert((link.to_string(), plink.to_string()));
                    } else if let Ok(parsed_link) = Url::parse(link) {
                        //url doesnt need to be parsed, add it to the hashset
                        link_hashset.insert((link.to_string(), parsed_link.to_string()));
                    }
                }
                None => continue,
            };
        }

        //if hashset is empty return a result of None
        if link_hashset.is_empty() {
            Ok(None)
        } else {
            //return a result of some
            Ok(Some(link_hashset))
        }
    }

and that is it for the html.rs file.

Now unto the client.rs file

first set up the usings:

use crate::html::HtmlRecord;
use bytes::Bytes;
use reqwest::header::USER_AGENT;
use url::Url;

Now set up the AGENT const under the usings, we'll be using the mozilla agent.

const AGENT: &str =
    "Mozilla/5.0 (platform; rv:geckoversion) Gecko/geckotrail Firefox/firefoxversion";

Now set up the Client struct

pub(crate) struct Client {
    pub client: reqwest::Client,
}

now lets set up the new function

impl Client {
    pub fn new() -> Self {
        Self {
            client: reqwest::Client::new(),
        }
    }
}

below the closing tag add the following method

pub fn replace_encoded_chars(body: String) -> String {
    body.replace("&lt;", "<")
        .replace("&gt;", ">")
        .replace("&quot;", "\"")
        .replace("&amp;", "&")
        .replace("&apos", "\'")
}

This is as the function states: replaces encoded characters to we get good results.

Now back into the Client struct, under new method add the following method:

    pub async fn fetch_html_record(&mut self, url_str: &str) -> Result<HtmlRecord, reqwest::Error> {
        let url_parsed = Url::parse(url_str).expect("cannot parse");
        let res = self
            .client
            .get(url_parsed.as_str())
            .header(USER_AGENT, AGENT)
            .send()
            .await?;
        let body = res.text().await.expect("unable to parse html text");
        let body = replace_encoded_chars(body);
        let record: HtmlRecord = HtmlRecord::new(url_parsed.to_string(), body);

        Ok(record)
    }

This gets the html and creates a record with it.

Next add the fetch_image_bytes method:

   pub async fn fetch_image_bytes(&mut self, url_str: &str) -> Result<Bytes, String> {
        let url_parsed = Url::parse(url_str).expect("cannot parse");
        let res = self
            .client
            .get(url_parsed.as_str())
            .header(USER_AGENT, AGENT)
            .send()
            .await
            .map_err(|e| format!("fetch image bytes failed for url {}:\n {}", url_parsed, e))?;

        let status_value = res.status().as_u16();

        if status_value == 200 {
            let image_bytes = res.bytes().await.expect("unable to parse html text");
            Ok(image_bytes)
        } else {
            Err("status on image call not a 200 OKAY".to_string())
        }
    }

Lastly add the fetch_string_resource methods. This grabs the css and the js for the webpage we are archiving.

    pub async fn fetch_string_resource(&mut self, url_str: &str) -> Result<String, String> {
        let url_parsed = Url::parse(url_str).expect("cannot parse");
        let res = self
            .client
            .get(url_parsed.as_str())
            .header(USER_AGENT, AGENT)
            .send()
            .await
            .map_err(|e| format!("fetch string resource failed for url {}: {}", url_parsed, e))?;

        let status_value = res.status().as_u16();

        if status_value == 200 {
            let string_resource = res.text().await.expect("unable to parse html text");
            Ok(string_resource)
        } else {
            Err("status on css call not a 200 OKAY".to_string())
        }
    }

The client was pretty easy. Now for the bread and butter, the archiver.
create the archiver struct

pub struct Archiver;

now create the save_page method. This uses everything we built upon to save the page to the base directory we provided

impl Archiver {
   async fn save_page(
        html_record: &mut HtmlRecord,
        client: &mut Client,
        base_path: &str,
    ) -> Result<String, String> {
        //set up the directory to save the page in
        let url = Url::parse(&html_record.origin).expect("can't parse origin url");
        let host_name = url.host().expect("can't get host").to_string();
        let mut url_path = url.path().to_string();
        let mut base_path = base_path.to_string();
        if !base_path.ends_with('/') {
            base_path.push('/');
        }
        if !url_path.ends_with('/') {
            url_path.push('/');
        }
        let directory = format!(
            "{}{}{}{}",
            base_path, host_name, url_path, html_record.date_time
        );

        //create the directory
        fs::create_dir_all(&directory).map_err(|e| format!("Failed to create directory: {}", e))?;

        //Get images
        match html_record.get_image_links() {
            Ok(Some(t_image_links)) => {
                assert!(fs::create_dir_all(format!("{}/images", directory)).is_ok());
                for link in t_image_links {
                    if let Ok(image_bytes) = client.fetch_image_bytes(&link.1).await {
                        if let Ok(tmp_image) = image::load_from_memory(&image_bytes) {
                            let file_name = get_file_name(&link.1)
                                .unwrap_or_else(|| random_name_generator() + ".png");
                            let fqn = format!("{}/images/{}", directory, file_name);
                            let body_replacement_text = format!("./images/{}", file_name);

                            if (file_name.ends_with(".png")
                                && tmp_image
                                    .save_with_format(&fqn, image::ImageFormat::Png)
                                    .is_ok())
                                || (!file_name.ends_with(".png") && tmp_image.save(&fqn).is_ok())
                            {
                                html_record.body =
                                    html_record.body.replace(&link.0, &body_replacement_text);
                            }
                        }
                    }
                }
            }
            Ok(None) => {
                println!("no images for url: {}", url);
            }
            Err(e) => {
                println!("error {}", e)
            }
        }

        //Get css links
        match html_record.get_css_links() {
            Ok(Some(t_css_links)) => {
                assert!(fs::create_dir_all(format!("{}/css", directory)).is_ok());
                for link in t_css_links {
                    let file_name =
                        get_file_name(&link.1).unwrap_or_else(|| random_name_generator() + "css");
                    if let Ok(css) = client.fetch_string_resource(&link.1).await {
                        let fqn = format!("{}/css/{}", directory, file_name);
                        let mut file = File::create(&fqn).unwrap();
                        if file.write(css.as_bytes()).is_ok() {
                            let body_replacement_text = format!("./css/{}", file_name);
                            html_record.body =
                                html_record.body.replace(&link.0, &body_replacement_text);
                        } else {
                            println!("couldnt write css for url {}", &fqn);
                        }
                    }
                }
            }
            Ok(None) => {
                println!("no css for url: {}", url);
            }
            Err(e) => {
                println!("error for url {}\n error: {}", url, e)
            }
        }

        //get js links
        match html_record.get_js_links() {
            Ok(Some(t_js_links)) => {
                assert!(fs::create_dir(format!("{}/js", directory)).is_ok());
                for link in t_js_links {
                    let file_name =
                        get_file_name(&link.1).unwrap_or_else(|| random_name_generator() + "js");
                    if let Ok(js) = client.fetch_string_resource(&link.1).await {
                        let fqn = format!("{}/js/{}", directory, file_name);

                        if let Ok(mut output) = File::create(fqn) {
                            if output.write(js.as_bytes()).is_ok() {
                                let body_replacement_text = format!("./js/{}", file_name);
                                html_record.body =
                                    html_record.body.replace(&link.0, &body_replacement_text);
                            }
                        }
                    }
                }
            }
            Ok(None) => {
                println!("no js for url: {}", url);
            }
            Err(e) => {
                println!("error for url : {}\n error :{}", url, e);
            }
        }

        //write html to file
        let fqn_html = format!("{}/index.html", directory);
        let mut file_html = File::create(fqn_html.clone()).unwrap();
        if file_html.write(html_record.body.as_bytes()).is_ok() {
            Ok(fqn_html)
        } else {
            Err("error archiving site".to_string())
        }
    }
}

Run through this code. Basically it follows the pattern of getting the resources and saving them to the body of the html document.

Now right above save_page, create the following method:

    pub async fn create_archive(
        &mut self,
        client: &mut Client,
        url: &str,
        path: &str,
    ) -> Result<String, String> {
        //create record
        let mut record = client
            .fetch_html_record(url)
            .await
            .unwrap_or_else(|_| panic!("fetch_html_record failed \n url {}", url));
        //save the page
        match Archiver::save_page(&mut record, client, path).await {
            Ok(archive_path) => Ok(archive_path),
            Err(e) => Err(e),
        }
    }

This gets the HtmlRecord and then extracts all its resources.

Now lastly in the main, add the mods

mod archiver;
mod client;
mod html;

Now add the usings

use crate::archiver::Archiver;
use crate::client::Client;

Now change the main to this:

#[tokio::main]
async fn main() {
    let url = "https://en.wikipedia.org/wiki/Rust_(programming_language)";

    /*
    change these two lines if you want to use an absolute path, or create the directory "/Projects/archive_test"
     */
    //this will grab your home directory
    let home_dir = dirs::home_dir().expect("Failed to get home directory");
    //make sure this directory exists:
    let custom_path = "/Projects/archive_test";
    //this is the absolute path to your home directory and the added directories to the spot you want to
    //add your archives to.
    let new_dir = format!("{}{}", home_dir.to_str().unwrap(), custom_path);

    //create the client and pass it to the archiver
    let mut client = Client::new();
    let mut archiver = Archiver;
    let path = archiver.create_archive(&mut client, url, &new_dir).await;

    //path of the archived site
    println!("{:?}", path);
}

change the values i used (the site url and the path (which needs to be absolute)).

now run the following command in the base directory of the project

cargo run

The output path is where your archive now resides.

Building a Syn Scanner in Rust

Al — Fri, 15 Dec 2023 22:12:37 +0000

Note:
Port scanning on hosts that don't like it when people probe them. They could actually bring legal consequences for port scanning them. Scan at your own risk. That being said, I've never heard of anyone getting arrested for port scanning.
i had to update some stuff bear with me

Thanks to chinoto for helping me.

Prereqs: having Rust installed

tldr : code

I'll be as brief as I can, and I hope to do a good write-up.

We'll be making a syn scanner in Rust. I love Rust, its a great language and I've had a lot of fun programming in it. You might have issues understanding whats going on if you're a beginner (or maybe as a pro if you think my code is janky). This code doesn't handle potentially filtered ports.

Lets start at what a syn scan should do.
The TCP handshake is the initial packet is a Syn packet which is like a request to connect. From there if the port is open receiver sends back a Syn/Ack. Then the sender sends an Ack packet. From there the connection starts.

In the syn scan, we send a Syn packet, and if the port is open we receive a Syn/Ack as a response. We never send back the last Ack packet to confirm connect. Instead we then send back an Rst packet otherwise the receiver of our Syn packet will continually send Syn/Ack to a timeout. We don't want to waste bandwidth.

Now that sounds super simple, and on paper it is. However to implement it is a tad bit more complicated.

Let's start out by making the new project. In the directory you have Rust projects in run

cargo new <Project Name>

replace project name for whatever you want to name it. I named it "syn-scan-practice"

Now navigate to the project folder and open up the Cargo.toml file and past the following dependencies in it:

pnet = "0.34.0"
rand = "0.8.5"
pnet_packet = "0.34.0"

As of 15/Dec/2023 these are the latest versions of the dependencies.

We are going to section this out in blocks, and I want it so we can do this with all the correct using statements right off the bat. So in main.rs paste the following usings at the top:

use pnet::{
    datalink::{Channel, DataLinkReceiver, NetworkInterface},
    packet::{
        ethernet::{EtherTypes, EthernetPacket},
        ip::IpNextHeaderProtocols,
        ipv4::Ipv4Packet,
        ipv6::Ipv6Packet,
        tcp::{ipv4_checksum, ipv6_checksum, MutableTcpPacket, TcpFlags, TcpOption, TcpPacket},
        Packet,
    },
    transport::{self, TransportChannelType, TransportProtocol, TransportSender},
};
use rand::{thread_rng, Rng};
use std::{
    net::IpAddr,
    sync::{
        atomic::{AtomicBool, Ordering},
        Arc,
    },
    thread,
    time::{Duration, Instant},
};

We will use all of these, and its a good amount of stuff to perform what on paper is a simple task.

Now lets get the consts out of the way, in main.rs copy paste the following into it under the using statements:

pub const IPV4_HEADER_LEN: usize = 20;
pub const IPV6_HEADER_LEN: usize = 40;
pub const ETHERNET_HEADER_LEN: usize = 14;

The length of the IPv4 header is 20, the IPv6 is double that. This gets added to the Ethernet header length which is 14. This all goes into the raw packet slice for building the packet.

Now lets set up the config struct, in main.rs copypaste this into it under the usings:

pub struct Config {
    interface_ip: IpAddr,
    source_port: u16,
    destination_ip: IpAddr,
    ports_to_scan: Vec<u16>,
    wait_after_send: Duration,
    timeout: Duration,
    all_sent: Arc<AtomicBool>,
}

The config will be mostly reused variables in a struct, this will later go into an Arc so we can pass it between threads. The meaning behind this is to potentially incorporate it in a larger project.

next, get the number generator up, this will be used for ephemeral the ephemeral port.

fn generate_random_port(min: u32, max: u32) -> u32 {
    let mut rng = rand::thread_rng();
    rng.gen_range(min..max)
}

Next we will want to implement new method for config, so in main.rs copypaste the following code:

impl Config {
    pub fn new(
        destination_ip: IpAddr,
        ports_to_scan: Vec<u16>,
        interface_ip: IpAddr,
        timeout: u64,
    ) -> Self {
        //get ephemeral port to use
        let source_port: u16 = generate_random_port(10024, 65535) as u16;
        Self {
            interface_ip,
            source_port,
            destination_ip,
            //set the kill flag, giving every syn packet a 500 millisecond roundtrip
            wait_after_send: Duration::from_millis(500 * ports_to_scan.len() as u64),
            ports_to_scan,
            //timeout just in case we need it.
            timeout: Duration::from_secs(timeout),
            //kill flag
            all_sent: Arc::new(AtomicBool::new(false)),
        }
    }
}

This is taking the destination IP and the ports to scan, and timeout just in case something happens receiving (something funky).

This is also setting up a source port to receive packets on a port that is Ephemeral. This also encompasses the "timeout" or the wait_afte_send (since it really does sleep the thread after send, but acts like a timeout).

Lastly, it has the "kill" flag, which is really the "all packets are sent" bool.

now lets set up the ability to get an interface:

//get the interface to establish a datalink channel
fn get_interface(interface_name: Option<String>) -> (NetworkInterface, IpAddr) {
    let interfaces = pnet::datalink::interfaces();

    //checks if interface exists, and if not it lists out available interfaces
    let Some(interface_name) = interface_name else {
        println!("Interface names available:");
        { interfaces.iter() }.for_each(|iface| println!("{}", iface.name));
        std::process::exit(1);
    };

    let interfaces_name_match = |interface: &NetworkInterface| interface.name == interface_name;

    if let Some(interface) = interfaces.into_iter().find(interfaces_name_match) {
        match interface.ips.first() {
            Some(ip_network) => {
                let interface_ip = ip_network.ip();
                (interface, interface_ip)
            }
            None => {
                panic!(
                    "cant get ip for interface \n {} \n interface: \n {}",
                    interface_name, interface,
                );
            }
        }
    } else {
        panic!("no interface named {}", interface_name);
    }
}

This will later be used and the arg to it is a command line arg at the start. This will also return the ip of that interface that will be used later.

Now lets set up the ability to get a socket to send the Rst and Syn packets:

fn get_socket(destination: IpAddr) -> Result<TransportSender, String> {
    match destination {
        IpAddr::V4(_) => {
            match transport::transport_channel(
                4096,
                TransportChannelType::Layer4(TransportProtocol::Ipv4(IpNextHeaderProtocols::Tcp)),
            ) {
                Ok((ts, _)) => Ok(ts),
                Err(e) => Err(format!("{}", e)),
            }
        }
        IpAddr::V6(_) => {
            match transport::transport_channel(
                4096,
                TransportChannelType::Layer4(TransportProtocol::Ipv6(IpNextHeaderProtocols::Tcp)),
            ) {
                Ok((ts, _)) => Ok(ts),
                Err(e) => Err(format!("{}", e)),
            }
        }
    }
}

This takes in the destination Ip to return a socket (or a TransportSender). You can play around with buffer sizes if you so like.

Now lets build out the build_packet function.

//build rst or syn packet
fn build_packet(
    tcp_packet: &mut MutableTcpPacket,
    source_ip: IpAddr,
    dest_ip: IpAddr,
    source_port: u16,
    dest_port: u16,
    syn: bool,
) {
    tcp_packet.set_source(source_port); //ephemeral port
    tcp_packet.set_destination(dest_port); //port being probed
    tcp_packet.set_sequence(0);
    tcp_packet.set_window(64240);
    tcp_packet.set_data_offset(8);

    //if syn, set syn flag, if rst set rst flag
    if syn {
        tcp_packet.set_flags(TcpFlags::SYN);
        tcp_packet.set_options(&[
            TcpOption::mss(1460),
            TcpOption::sack_perm(),
            TcpOption::nop(),
            TcpOption::nop(),
            TcpOption::wscale(7),
        ]);
        tcp_packet.set_urgent_ptr(0);
    } else {
        tcp_packet.set_flags(TcpFlags::RST);
    }

    let checksum = match (source_ip, dest_ip) {
        (IpAddr::V4(src), IpAddr::V4(dest)) => {
            ipv4_checksum(&tcp_packet.to_immutable(), &src, &dest)
        }
        (IpAddr::V6(src), IpAddr::V6(dest)) => {
            ipv6_checksum(&tcp_packet.to_immutable(), &src, &dest)
        }
        _ => panic!("cant create socket in get_socket"),
    };
    tcp_packet.set_checksum(checksum);
}

You are free to read up on the Tcp field values, the ones on this are tested and work, at least for me. If you have questions as to what they do, I suggest looking over this TCP header explanation.

now for the get_buffer function

fn get_buffer(config: &Config) -> Vec<u8> {
    let header_length = match config.destination_ip {
        IpAddr::V4(_) => IPV4_HEADER_LEN,
        IpAddr::V6(_) => IPV6_HEADER_LEN,
    };
    let vec: Vec<u8> = vec![0; ETHERNET_HEADER_LEN + header_length + 86];
    vec
}

this is to reduce redundancy, it builds out a buffer in case of a IPv4 packet or an IPv6 packet

Now lets build out the functionality to handle the tcp packets:

fn handle_tcp<'a>(
    ip_payload: &[u8],
    config: &Config,
    ip_addr: IpAddr,
    buffer: &'a mut [u8],
) -> Result<Option<MutableTcpPacket<'a>>, String> {
    let tcp_packet =
        TcpPacket::new(ip_payload).ok_or_else(|| "Failed to create TCP packet".to_string())?;

    let mut rst_packet = MutableTcpPacket::new(buffer)
        .ok_or_else(|| "Failed to create mutable TCP packet".to_string())?;

    if tcp_packet.get_destination() != config.source_port {
        return Ok(None);
    }

    if tcp_packet.get_flags() == TcpFlags::SYN | TcpFlags::ACK {
        println!("port {} open on host {}", tcp_packet.get_source(), ip_addr);
        build_packet(
            &mut rst_packet,
            config.interface_ip,
            config.destination_ip,
            config.source_port,
            tcp_packet.get_source(),
            false,
        );
        Ok(Some(rst_packet))
    } else if tcp_packet.get_flags() == TcpFlags::RST {
        Err(format!(
            "misc flag {} on port {}",
            tcp_packet.get_flags(),
            tcp_packet.get_source()
        ))
    } else {
        Ok(None)
    }
}

This incorporates a lifetime tied to the input buffer as the output packet contains a reference to the input buffer

Now lets build out the send_packets function:

fn send_packets(config: &Config, mut sender: TransportSender) {
    for destination_port in config.ports_to_scan.iter() {
        //get buffer to send
        let mut vec: Vec<u8> = get_buffer(config);
        //create packet
        let mut tcp_packet =
            MutableTcpPacket::new(&mut vec[..]).expect("Failed to create mutable TCP packet");
        //build packet
        build_packet(
            &mut tcp_packet,
            config.interface_ip,
            config.destination_ip,
            config.source_port,
            *destination_port,
            true,
        );

        //send packet
        if let Err(e) = sender.send_to(tcp_packet.to_immutable(), config.destination_ip) {
            eprintln!("Error sending packet: {}", e);
        } else {
            println!(
                "sent syn packet to port {} on host {}",
                destination_port, config.destination_ip
            );
        }
    }
    //sleep to change the all_sent flag
    thread::sleep(config.wait_after_send);
    //mark to true to kill receive_packets loop
    config.all_sent.store(true, Ordering::SeqCst);
}

This takes the config struct and a transport sender to send the packets. It builds a syn packet and sends it using the socket or the TransportSender. Look over the code and I will do my best to comment the source code on github to explain things that may not be apparent. Feel free to leave a comment if you don't understand whats going on in it.

Now the tough part, receive_packets function.

fn receive_packets(
    config: &Config,
    mut sender: TransportSender,
    mut rx: Box<dyn DataLinkReceiver>,
) {
    let start = Instant::now();
    //loops over received packets, may not be all our packets
    loop {
        //we got a packet
        match rx.next() {
            //we got a packet
            Ok(packet) => {
                //Build from bottom up the stack, starting with ethernet
                let eth_packet = EthernetPacket::new(packet).unwrap();
                //get the type of packet riding on ethernet packet
                match eth_packet.get_ethertype() {
                    //type of IPv4, we want this
                    EtherTypes::Ipv4 => {
                        let ipv4_packet = Ipv4Packet::new(eth_packet.payload()).unwrap();
                        //if next layer up isn't tcp, we don't care
                        if ipv4_packet.get_next_level_protocol() == IpNextHeaderProtocols::Tcp {
                            //get buffer to have a lifetime outside the function, done for ownership reasons
                            let mut buffer = get_buffer(config);
                            //develop rst packet to send
                            let rst_packet = handle_tcp(
                                ipv4_packet.payload(),
                                config,
                                ipv4_packet.get_source().into(),
                                &mut buffer,
                            );
                            //rst_packet is a Result<Option> so this unwraps it all
                            match rst_packet {
                                Ok(rst_packet) => {
                                    if let Some(rst_packet) = rst_packet {
                                        let _ = sender.send_to(rst_packet, config.destination_ip);
                                    }
                                }
                                Err(e) => {
                                    //prints too much
                                    println!("error {:?}", e)
                                }
                            }
                        }
                    }
                    //type of IPv6, we want this
                    EtherTypes::Ipv6 => {
                        let ipv6_packet = Ipv6Packet::new(eth_packet.payload()).unwrap();

                        //if next layer up isn't tcp, we don't care
                        if ipv6_packet.get_next_header() == IpNextHeaderProtocols::Tcp {
                            //get buffer to have a lifetime outside the function, done for ownership reasons
                            let mut buffer = get_buffer(config);
                            let rst_packet = handle_tcp(
                                ipv6_packet.payload(),
                                config,
                                ipv6_packet.get_source().into(),
                                &mut buffer,
                            );
                            match rst_packet {
                                Ok(rst_packet) => {
                                    if let Some(rst_packet) = rst_packet {
                                        println!("{:?}", rst_packet.get_source());
                                        println!("{:?}", config.destination_ip);
                                        let _ = sender.send_to(rst_packet, config.destination_ip);
                                    }
                                }
                                Err(e) => {
                                    //oops
                                    println!("error: {:?}", e);
                                }
                            }
                        }
                    }
                    //we dont care, do nothing so we can check breaking condition
                    _ => {}
                }
            }
            //print out if there is an error receiving a packet
            Err(e) => {
                println!("error while receiving packet: {:?}", e)
            }
        }
        //check if all_sent flag is true
        if config.all_sent.load(Ordering::SeqCst)
            //check if timeout is hit
            || Instant::now().duration_since(start) > config.timeout
        {
            break;
        }
    }
}

I commented a lot on the code to clarify some things, let me know if it is unclear.

Now set up the main

 fn main() {
    //setting up values for new config and an interface to send into receive_packets
    let interface_name = std::env::args().nth(1);
    //set up var to use, make sure its in scope
    let destination_ip: IpAddr;

    if let Some(ip) = std::env::args().nth(2) {
        match ip.parse::<IpAddr>() {
            Ok(ip) => destination_ip = ip,
            Err(err) => {
                panic!("error parsing ip address {}\n error: \n {}", ip, err);
            }
        }
    } else {
        panic!("no ipv4 or ipv6 address provided")
    }

    //change to desired ports
    let ports_to_scan: Vec<u16> = vec![443];

    //get interface and the interface_ip
    let (interface, interface_ip): (NetworkInterface, IpAddr) = get_interface(interface_name);

    //timeout value if op isnt working correctly, going to be in secs.
    let timeout: u64 = 10;

    //config
    let config: Config = Config::new(destination_ip, ports_to_scan, interface_ip, timeout);

    //create the arcs to send among other threads
    let config_arc = Arc::new(config);
    let config_arc_clone = config_arc.clone();

    let rx_builder = thread::Builder::new().name("receive_thread".to_string());
    let tx_builder = thread::Builder::new().name("send_thread".to_string());

    //set up senders for sender syn packets (sender) and rst packets to close the connection (receiver)
    let syn_sender: TransportSender =
        get_socket(destination_ip).expect("Failed to create tx_sender");
    let rst_sender: TransportSender =
        get_socket(destination_ip).expect("Failed to create rx_sender");

    //get interface to receive syn packet responses
    let rx = match pnet::datalink::channel(&interface, Default::default()) {
        Ok(Channel::Ethernet(_, rx)) => rx,
        Ok(_) => panic!("Unknown channel type"),
        Err(e) => panic!("Error happened {}", e),
    };

    //receiving packet thread
    let rx_thread = rx_builder.spawn(move || {
        receive_packets(&config_arc, rst_sender, rx);
    });

    //sending packet thread
    let tx_thread = tx_builder.spawn(move || {
        send_packets(&config_arc_clone, syn_sender);
    });

    //joining the handles
    let _ = rx_thread.expect("receive failed").join();
    let _ = tx_thread.expect("send failed").join();
}

Again, I've attempted to comment it out to make it more clear. Some things you can change are the destination_ip or the ports_to_scan.

So now do a cargo check to make sure if it is all there.
And if that's all okay, run it
(for linux/mac)

sudo cargo run <Interface Name> <IP>

Replace with the name of the interface you want to use, same with IP.
If you run
(for linux/mac)

sudo cargo run

you will get a list of interfaces available

If your on windows, I think you can run

cargo build

and right click the binary and "run as administrator" it. If that doesnt work let me know, I'm not a windows user so I can't trouble shoot issues on it at the moment.

Let me know if you have issues, I ran through the tutorial quick and I came up with no issues. But I can miss stuff.

This is how to build a syn scanner in Rust, if you have any ideas, link up on the comments or github.

Some ideas for future are some things like figuring out how to reuse the buffer, adding the ability to input ip from command line (can take v4 and v6), take in custom port ranges || subnets || single ports, etc.

Thanks