15 minute read.
- Acknowledging Risk
- Understanding the principal of low hanging fruit
- Creating a security policy
- Server Security
- Email attackers
- User Roles and Permissions
- Data Sanitizing
- Request Throttling
- Physical Security to Augment Digital Steps
- Self Reflection
I wrote this post to try to answer some of my own questions about what modern database security in 2021 looks like, especially if you’re in the tech field and looking into creating or supporting your own database management system.
While the scenarios discussed can be applicable to a wide variety of domains and specialities, most of the examples will pertain to web, web accessible applications, and those with self-hosted servers.
At the time of writing this (early 2021), the COVID-19 pandemic has decimated most of the world’s ability to freely travel, and thus there’s been a sharp up-tick in the creation and desire for online solutions. Developers has mobilized to create a variety of web stores, app portals, and even SMS-based platforms to meet the needs of clients without requiring them to visit in person stores.
But with a rise in both product and service availability online, and demand from consumers, we should also expert a rise in malicious actors willing to exploit the system for personal gain. Cyber crime is on the rise. The year 2020 saw a marked increase in online criminal activity in the order of billions of dollars in costs to businesses, not to mention the consequences of serious data breaches for consumers.
So let’s dive right in.
It may sound paranoid to say, but I genuinely think that everyone should approach software development in 2021 with the approach that bad actors will make at least some attempt to compromise your operations. Some studies estimate that the average cost of cyber attacks on business is up to $2.6 million.
Unfortunately, the more successful your business becomes, and the more notoriety you gain, the greater the chances that you’ll receive unwanted notice from cyber-criminals, whether as part of some wide scale attack, or specific intent.
“I’m too small to be targeted,” is a phrase you might say to yourself.
Let’s swiftly get that notion out of our minds.
Like real life thefts and ransoms, being small makes you more likely to be targeted, not less. Your average thief does not rob multi-million dollar houses; the risk reward profile makes small to medium sized businesses more attractive.
They tend to have much worse security to prevent attack, and less resources to combat an attack after it’s happened.Plus, since this is all digital, your average malicious entity would much rather deploy ransomware on hundreds of small to medium sized businesses, than chase a big whale that had the potential to bring down their operation. Growing business are extremely vulnerable and should always place themselves in a high risk category.
“The question isn’t, ‘Are there bears nearby?’ There’s definitely bears nearby. That’s just the price to pay for coming to the woods. They’re here and there’s nothing you can do about it. Your job is just to make it as hard as possible for them to get you while you’re here.”
This photo isn't real, but it got my point across way too well not to use it.
You don’t need to be the first guy, but you definitely don’t want to be the last guy.
While I’d love to be able to report that a perfect defense exists, and some combination of money and technology can keep you 100% safeguarded, but unless your entire business lives in a safe that you never open, that’s just not the case.
What I learnt is rather than try and create some perfect, impregnable system, your time is better spent following a Pareto Principle approach; an iterative process where you continuously adopt the best solutions for your setup, balance cost and expense vs effect.
The picture above, while hilarious, perfectly illustrates the type of environment we’re dealing with in today’s digital society:
the bear (cyber-attackers), are faster and stronger than you are. Attackers will always be one step ahead of defenders. As defenders, we can only react and guarded against the threats we know. Attackers are constantly creating threats we Don’t know. Once a new threat is discovered, we scramble to fix things before it’s too late. Sometimes we’re successful, sometimes we aren’t.
As morbid as it sounds, your goal isn’t to be most secure platform in the world; it’s just to be the least, attractive target for attack.
Unlike getting picked as someone’s pacing buddy on the annual company hike, you never want to be a hacker’s first choice. Attacks are best experienced vicariously; we watch when they happen to others and make sure it doesn’t happen to us.
What does best solution mean? I tend to break it down with the following statement;
“What is the cheapest solution (in terms of cost, development cycles, dev op hours, troubleshooting, etc) to move a vulnerability in my system one tier down from where it currently exists.”
I tend to classify my risks in to three categories:
Critical Weaknesses: Obvious, easily exploitable weaknesses that could instantly bring down your organization or result in a data breach. These are also things that are easily discoverable. Invalid/Non existent security certifications, unsafe form validation, open API access. Every day these continue to exist you enterprise is moments away from an unrecoverable failure.
Medium Risk: These are the types of problems that are looming threats, and could easily escalate if left unchecked. Exposed/Unrestricted database access within the network, weak passwords for sensitive data, lack of firewall/antivirus software for your servers. They may not seem like immediate problems, but are essentially ticking time bombs, and it’s still a matter of when, not if, that you’ll regret not addressing them
Low Risk: This is where you’ve covered all the basics, and you’ve basically got a few holes left in your system for rare events. Should you address these at some point? Yes. But nothing here should be an open door and would either require a large amount of insider knowledge or effort on the side of the attacker.
So your strategy becomes dealing with critical and medium weaknesses as quickly as possible and not falling into a fall sense of security because you’ve done a bunch of updates to low risk issues. It doesn’t matter if you’ve got the best home security system money can buy if you’ve left your door open.
Edited out of this image are the tears the security admin shed at having to write more that two pages of text.
You should have a security policy. That’s it, that’s the tweet. Every business or product should have a security policy. There’s lots of good resources on how to make one, but any policy you make needs to secure the bare minimum:
What data do we collect and how much security does it require (credit card info needs lots of security, favourite power ranger, not so much)
Where do we keep our data and how difficult is it to access?
Who has access to what parts of the system?
What do we expose to the outside and through what channels do we expose it?
What 3rd party applications do we depend on and how secure do we think those are?
What's my company’s risk profile?
The more data you collect, the higher your risk profile
The more users you have, the higher your risk profile
Financial, Health, and location data are especially high risk
As we’ve talked about earlier, the best security is a black box; a network completely isolated from external networks (and the internet) is the only true way to prevent attacks via that vector.
Unfortunatly, your data doesn’t exist in isolation. Even if it has no direct connections, if your database and DBMS live on a network with internet access, then your data is somewhat exposed.
Keep your databases hidden on the network. No user who doesn’t need access to the DB should even be able to perceive it on the company network.
Limit the ports that can access your database server (this shuts down a large swath of potential attack vectors
Be aggressive in your firewall usage for your critical data. Always follow the principal of minimal access. Every port and connection should be closed unless you explicitly open it. Every request should be untrusted unless specifically trusted. Every user should be permissionless unless explicitly granted permission.
Before we get too deep, I need to note that you can do all the protections in the world, but the majority of cyber attacks still come from basic phishing attacks through email.
Anyone in your organization who willingly clicks a link to a malicious website or downloads an attachment from an unrecognized source has the capability to introduce malware and ransomware to the machine they used to access that link.
Beyond setting up limitations on downloads for browsers on work machines, the best you can do is educate all your users on proper email safety practices.
Always check to ensure you’re receiving emails from a qualified sender (I highly suggest using tools that can identify addresses that operate in the same user group as yours).
A common phishing tactic is to create an address that closely mimics one within your network, e.g.
firstname.lastname@example.org, with the attacker hoping you will download and run the attached malware before you notice that that address actually isn’t valid. Be vigilant.
Once a user has downloaded something design to compromise the network, that’s where our safeguards in part one can kick in to ensure that the most that can happen is that files on the local machine are the only ones at risk and the rest of the company network (especially our key database stores) remain safe.
As I said earlier, one of the primary purposes of this article is to help folks who create, maintain and utilize database management systems, especially those of us who has to store sensitive user data. Once someone has managed to breach your firewalls and network permissions, the final layer is your DBMS itself. Similar to the rules in safeguarding your network, you want to be hyper specific in what access your server allows. Most DBMS systems allow you to fine tune exactly what each authorized user is allowed to do.
- As a general rule, no one but the system administrator should have full permissions on the system.
- Enforce strict password policies but avoid the common pitfall of having such frequent password overturn that users start being lazy about their password choices.
- Create dedicated profiles for any apps or services that have access to your database.
- When possible, always use stored procedures or other predefined query systems. Do not let any user have the ability to make unrestricted queries to the database.
- Periodic backups are key. Automated backups. Telling yourself that you (or whoever is in charge of the db) will remember to do manual backups is a falsehood.
- Ensure that all your backups are Not kept on the same server instance as the main database. This completely defeats the purpose of the backups.
- Backup automation should be one way; getting data back from deep backups should be something that at the very least requires an explicit trigger from your system administrator/security specialist.
Any data coming in from external sources should be treated as a potential vector for attack. Under no circumstances should you ever return raw input from your forms and web views into your database. Typical strategies for ensuring the right data makes it to your application involve a mix of sanitizing, hashing, and streamlining. I want to highlight some of the most common ones below.
Sanitize your inputs. This means ensuring that the text your users type in is free of any erroneous or malicious text that might compromise your system (or just be invalid data). You have the option of sanitizing inputs in your front end code immediately in the handlers for input elements; you can also do in your web server. My suggestion; do both. Things to typically guard for:
SQL injection by adding commands that can be executed by an sql based db:
- null and oversized strings
- strange symbols and special characters
- Anything that could be interpreted as a path/file extension/internal command
- Do not let unqualified strings pass from your users keyboards to your database. You will regret it.
Use stored procedures - A stored procedure is a pre-written sql procedure or query that can be called by a user/service, and can work as an api to your DBMS. Rather than allow your application to pass queries directly to your database, you can allow your app access to stored procedures only, sealing up the apps ability (or anything impersonating the app) to ever issue rogue requests to the server.
Hash sensitive data - Do not store raw passwords in your database. This is another “that’s the tweet moment.” The best way to keep user data from being compromised is to Not Store It. The second best way is to use various hashing, salting, and encryption techniques to make sure that if anyone does get access to data they shouldn’t, they can’t actually use it.
Store Less Data - Yes, you heard me. Store less data. I get it, storage is cheap nowadays and we all feel like not keeping every bit of data our users are willing to give up is leaving money on the table, but you should always be weighing the risk vs reward of storing sensitive user data. Maybe you don’t need to store data you don’t currently need as a “just in case”. The less data you keep, the less of a target on your back.
DDOS (Distributed Denial of Service) attacks suck. Without getting into the details, DDOS attacks involve bombarding a server with an overwhelming amount of requests, which can cause the server to crash, or break in cryptic ways, accidentally revealing data that should not be available to the attacker. It can also be used to hold an organization hostage, as typically regular users are unable to use the API while it is under attack.
And if you haven’t been paying attention to cybersecurity news recently, DDOS attacks are on the rise, both in frequency, and in severity. The original attacks sent a few hundred requests per second. Currently tech employed by bad agents allow them to send millions of requests per second. DDoS attacks can happen at any time, and there’s not much you can do to stop the attack from happening. But there are strategies to stop the attack from harming you.
Request throttling refers to purposefully limiting the rate of requests your servers process at any given time, and can be a good tactic to prevent and limit the damage of DDOS attacks. Many hosting platforms have anti DDoS policies and methods and it’s worth spending some time finding a host prepared to defend against such attacks, and if self hosting, having a good strategy can save a lot of heartache in the long run.
Ultimately, developers are people. People make mistakes. People have deadlines and stressors and we get tired. Most, if not all of us want to be a good job, but we can’t let to help some subtle things slip by the wayside. So I think it’s always good to go over the small quick wins that can avoid egg-on-face moments down the line.
Strong password policy - There’s no excuse for weak password policies in 2021. And please opt for stronger password vs frequently changing ones. I find users bristle at 3 month password changes and begin to find ways to subvert the system. 6-12 months seems ideal but ultimately longer passwords are better passwords.
Limit jump drive usage on work machines. If this isn’t already a policy at your organization you may get a lot of push back. It’s worth pressing on though. Jump drives are error-prone at best and a vehicle for espionage and malicious software at worst. For employees that truly need to access work from outside their designated machines, allow secure ssh sessions before copying onto external storage devices.
Related to above, do not let employees use personal computers to access the server. The risk is almost never worth the reward. C suite employees tend to push back on this the most. Attempt to hold strong. You have almost zero control over what happens once a personal machine has access to sensitive data. They could already be compromised. Again, use secure ssh based channels for file access, not file transfer.
Do not do indefinite logons. In the recent Capital riots, images showed the attackers observing unsent emails and other sensitive documentation left available on the machines of government employees. Devices left unattended should log out inactive users sooner rather than later.
Access tracking and logs. There should always be a record of what users are active at what times, the data they request, and the changed they make to the system. Even in benign circumstances, this can prove useful.
All the actions just described are relatively easy to implement, but combined seal of many of the less glamorous ways that servers and databases get compromised. It’s the type of work that can pay dividends in the long run and great bang for your buck in terms of effort vs reward. I highly encourage anyone who cares about database security to check off some of these boxes.
*A small note on privacy. Others may disagree, but I personally believe that employees shouldn’t keep secrets while on company hardware. In that vein, I think companies have zero business tracking employee actions outside of designated work environments. Company emails and company file and process access are within the purview of company security and risk mitigation. Employee’s homes, and personal habits in their lives are not. Keep the lines and distinctions clear.
Finally, I want to spend some time on what is often enough the biggest weakness in an organization’s security; the security admin themselves.
It is terrifying how often I see security experts apply all the following rules to everyone but themselves.
We see the news of companies and organizations that get hit and all publicly shake our heads and say, “could never be me”.
But most likely it could. And most likely it will. Knowledge and effort can go a long way, but a dash of humility doesn’t hurt either.
Write your processes down. Designate a secondary for your critical operations. Prepare a plan in case of your sudden illness. Never assume you have all the answers, and never stop learning.
Most importantly, be kind to yourself and those around you. It’s never been harder to keep information safe and secure; we’re all just trying to do our best.
Hopefully this article can give folks a solid starting point to assess themselves and their organizations.
Thank you for your time.