I Don't Know Crap About Security (Or At Least I Didn't)

This was originally posted on my blog.

Recently, I was involved in a project that required doing some serious locking down of an application prior to an extensive audit (with potentially millions of dollars on the line). All the buzzwords were there: "pen testing", "red teams", and "everything depends on this". This wasn't going to be a simple, one-click fix. It was going to require me to pop a few Mountain Dews, dress in all black, and play pretend as a hacker:

Needless to say - I was terrified being tasked with this responsibility, because as the title of this post would indicate: I don't know crap about security (or at least, I didn't).

You Don't Know What You Don't Know

I would say that I'm probably in the same boat as most developers, in that, security is seldom the first thing that comes to mind when building an application (if I even think of it at all). Typically, I tell myself things like the following:

• "This is an on-prem application, so I shouldn't need to worry about anyone hacking it."
• "I'm writing this in {insert-framework-here}, so I'm sure they've taken care of this for me."
• "I'll do it later..."

The problem is that often, all of these have a single thing in common: you aren't really doing anything to ensure that your application is protected. You might rely on your favorite framework or justify the excuse for putting things off for whatever reason - but the security of your application should be treated just like your code, in that, you should actually test it out.

Your application might be fine. You may have followed all of the best practices and treated security as a first-class citizen within your app, but you don't know what you don't know, and until you try and break things, you won't.

Breaking Things

In my particular case, the application I was trying to battle-test, was an older application that's been around for several years, so it isn't using the latest and greatest technologies and frameworks. If you've been a developer for a few years, I'm sure you can imagine exactly the type of application I'm talking about. A little WCF here, a little Web API there, maybe some ASPX pages sprinkled on top of a SPA, like I said - you probably know what I'm talking about.

I work with an incredible team of folks: awesome, passionate developers that make me feel dumb every day, but I know that we aren't perfect. Smart people make mistakes too, it's just a part of the game. Security related issues are often less of actual bugs and more of something that isn't given adequate thought (or time).

It's because of this, that we have to burn things down in order to build them up. We need to grab the biggest bat lying around the office and repeatedly whack our application so that we can see what's left in the remaining pile of rubble:

Once you see what areas of your application crumbled to pieces with ease, you'll know where to start trying to make improvements. While there are plenty of ways to do this, automation is going to be our friend here. Don't try and invent your own security testing platform, you'll just make things worse. There are plenty of openly available tools (as well as a few commercial ones that offer free trials) out there that'll do all the heavy lifting, such as:

• Burp - The gold standard in the world of penetration testing, Burp is the real-deal used by security professionals across the globe. It can function as a proxy to run your traffic through and scan it in real-time, quickly identify potential security concerns, and provide you with fine-grain control over what you want to try and exploit within your application. It may not be the prettiest tool ever, but it's extremely powerful, extensible, and worth exploring.
• NDepend - A great static analysis tool that integrates into Visual Studio that can help identify bad patterns, dead code, and much more. It's an invaluable tool both in and out of the security space.
• Puma Scan - A Rolsyn-powered analysis tool that integrates with Visual Studio as an extension that can help easily identify security vulnerabilities like SQL Injection, XSS, and more within your code (and help you fix it). It's free, but it also has a commercial version that you can incorporate as part of your build process to help keep your application secure.
• ZAP - Another great community-based tool from OWASP, which is very similar to Burp and is a great alternative if you can get ahold of Burp (or don't need all of the features that come along with it).
• Fortify - This was a fantastic tool that I just so happened to have access to. It's incredibly powerful and can automatically crawl your site/application (even SPAs) using highly-configurable compliance levels depending on what you need.

Having all these wonderful tools at your disposal is great, but it can be overwhelming. You probably need to give some thought to what your own concerns are with your application and focus on those, otherwise, you can consider performing a broad scan of the application and then dig into things a bit deeper (i.e. breadth-first then depth-first).

You Need A Plan

You need to dedicate some thought into how you are going to start securing your application, determine what's important to you (and your organization), and then triage the critical items that pop up.

Every application is different. There's no silver bullet, one-click scan that will magically find everything wrong with your application (although some of the commercial-grade tools come pretty close). I'll provide an example of a workflow I used to help turn a pretty porous application into something that could survive a high-level penetration test:

• Set Expectations - This process can be extremely time-consuming. Some scans can take days to run, some might fail, machines might die, environments can get wrecked. You need to dedicate some serious time to this process if you want to make it worth your while, and you need to have a good idea of what you are trying to improve (e.g. get rid of all of the Critical Vulnerabilities, remove any XSS vulnerabilities, pass a HIPAA compliance test etc.)
• Create A Workflow - In some instances, an automated scan might not be enough. If you have a pretty advanced SPA, some crawlers might just give up, so you have to get clever. Consider setting up a proxy such as Burp to capture all the traffic against some shared test environment. This will give you real-world requests to feed into some of these tools so that they can better access your application and try to break it.
• Set Up A Testing Environment - Generally penetration testing will ruin a development environment and any associated databases, so if you can create a "throwaway" place to perform these tests, then your life will be much, much easier. Tools like Docker can help with this, so that each time you need to perform a scan, you could hydrate a database within a container, point your penetration testing tools at it, and then blow it away after you are done.
• Define A Baseline - Consider running a very throughout vulnerability scan of your application. Crank up the settings to their highest and just let it run. It might take hours, days, or in excess of a week, but it'll help you get an idea of where you are in the eyes of the security gods and what you need to work on.
• Focus on Severity - Most of the tools mentioned earlier will assign severity levels to any vulnerabilities that you come across, and it's important to weigh these accordingly. A handful of critical-level vulnerabilities will likely trump hundreds of not thousands of trivial ones, so I know it might "feel good" to see your numbers drop significantly, if you aren't tackling the major issues, you might want to consider why you are doing this in the first place.
• Mitigate, Rinse, Repeat - After you've identified some of the issues that are concerning, start to triage them. Many of the tools out there will provide severity levels, which are an excellent guide. Once fixes are in place for these issues, repeat the process and re-scan to ensure that the tools no longer see the vulnerabilities.

After several iterations, you should see the number of vulnerabilities in your application drop. Major issues might require some serious engineering and refactoring of code, processes, and more, but they are well worth it when some super villain elects to attack your system and put it in the headlines.

The real challenge though, is staying protected.

Building Habits

Let's say that you've dedicated hundreds of hours to risk your application of security vulnerabilities. Things like XSS attacks, SQL Injection, and CSRF are all things of the past... or at least, they were the last time that you scanned your system.

In a world of continuous deployment/integration, maintaining security standards and code quality can be challenging. Large teams, minuscule iteration cycles, and high-pressure deadlines can easily cause security to get thrown by the wayside in order to get that next version shipped. This is why it's so important to build good habits, as a team and leverage tools that can help ensure you stay productive and protected.

Some things to consider helping with these efforts include:

• Leverage Static Analysis Tools - There are plenty of tools out there, including some of those that I mentioned earlier like Puma Scan, which can integrate with your IDE and provide development and build time errors and warnings that can help you and your team stop vulnerabilities before even writing them.
• Build Pipeline Gates - Nearly every build pipeline worth its salt has support (either baked in or available through extensions or plugins) to disallow code from being committed if it results in a broken build, or in the context of security, any vulnerabilities that your team has decided cannot be allowed into production.
• Automation and Containerization - Make this process automated, scripted, etc. Consider spinning up something at the beginning of a sprint if you do agile, gather proxied data through shared test environments, at the end of the sprint, run a scan against all the traffic and see what comes up (to ensure no new vulnerabilities were introduced)
• Share the Knowledge - Consider having tech talks where you can tackle a specific type of vulnerability, how to identify it, what some of the dangers of it are, so that your developers are better informed and know how (and why) they should avoid it.

Security is a hugely broad topic, akin to architecture, in that you can't try to fix all the issues at once. Focus on building good habits and tackling important problems as they arise, and you'll find that the security of your applications will become less triage and more maintenance, which is a great position to be in. You'll know more of what you know (and what you don't) after putting some of these pieces in place, which should help put your product, your company, and your engineering team's collective minds at ease.

So, I'll leave you with this - you don't need to be a security expert to start caring about security. If you've been putting security off, don't; start making baby steps to put the pieces of the puzzle together. I can honestly say as an experienced engineer that I dove into this pool head-first and lived to tell the tale (and write this post), so I have no doubts that you can too. Don't let "not knowing crap" about security be the biggest reason that you aren't caring about security. It's an area where a small amount of effort can yield huge dividends and your product, your company, and your entire engineering team will be better for it.