If you have a website, then chances are that your website is vulnerable to all sorts of attacks. And the bigger and more complex your site is, it's harder to keep track of potential security glitches.
This is why you need a Web Application Security Scanner, or just a Scanner. At first sight, the Scanner's job is easy. It scans your web app and generates a report highlighting the security complications it found so that you can mitigate them.
The problem is that since custom web apps can be put together in a million different ways, finding security issues with a Scanner is not always reliable. Sometimes the Scanner will find and report a vulnerability within your web app, and sometimes it won't.
So just how good are some of the Scanners out there - and how can you know if you're getting your money's worth?
To find out, I created a web app that has three big security flaws. These imperfections should ideally be detected and reported by all Scanners.
The three flaws in the web app are:
One - Sensitive information as source code comment:
<!--the username for our most sensitive data is Summer2018 with the password MegaSecret-->
Two-Sensitive data exposure:
The root folder has a file called passwd with the following content:
tom:x:1000:1000:Vivek Gite:/home/vivek:/bin/bash
Three-Code injection:
The website has a form where the user can fill in her e-mail address and press submit - and the e-mail address will be echoed below the submit button like so:
The problem is that the user input is not sanitized. So if anyone was to enter Hello!!!
And it's also possible to insert JavaScript and all sorts of other unpleasant stuff.
Meet the scanners
Patronus.io: They claim to be an -"automated security solution for your website". Sounds good to me but let's see if it's true shall we?
Detectify: Say that they "scan your website for security issues crowdsourced by 150+ white-hat hackers". Sounds pretty underground and cool doesn't it? But is it all talk?
Tenable: -They "…safely, accurately and automatically scan your web applications, providing deep visibility into vulnerabilities and valuable context to prioritize remediation". Great, but does it work?
Round one - Discovering sensitive information as source code comment
No -Patronus.io
Yes - Detectify
No -Tenable
Round two - Discovering sensitive data exposure
No - Patronus.io
No - Detectify
Yes - Tenable
Round three - Discovering Code injection
No - Patronus.io
Yes - Detectify
No - Tenable
So who won?
The winner
The Detectify scan took by far the longest to complete. But it sure was worth the wait. Detectify managed to find both the code injection vulnerability and the source code comment containing the username and the password. Detectify failed to find the passwd file but I can live with that shortcomming
Runner-up
Tenable had no problem finding the passwd file in the root directory. However, the code injection vulnerability flew under the radar as did the username and password exposed in the HTML source code
Dead last
Patronus on the other hand is surprisingly week. Their scanner did not find any of the three vulnerabilities. I suggest you take your money elsewhere as this product is only going to give you a false sense of security. Security tools that are this weak should be banned altogether - we simply deserve better.
Conclusion
Automated scanner aren't very good. Sure, they can catch a low hanging fruit or two - but they are still way behind manual penetration testing. Scanners should therefore only be seen as a small piece of your security puzzle. Because if you rely on scanners alone for your web app security posture you might be in for a nasty surprise.
Top comments (7)
Obvious solution: Stop misusing your web browser as a "platform"! If your site has become a "complex application", chances are that you should write a desktop application instead. No desktop application written by you could ever be nearly as insecure as your web browser.
There's some truth to that of course. But sometimes you just get a website/service/whatever dumped in your lap with a -"hey, keep this spaghetti monster secure now ok?".....and you need to figure out a decent way forward.
That's true. Usually, OWASP ZAP should be good enough for that if you know wtf you are doing.
But sometimes a complete rewrite is still the better solution.
I agree x 2
Hi Robert,
this is Jens from patronus.
Thank you for testing and evaluating our tool.
I have to add the information, that you has executed your tests with the free package. This package does not contain the penetration test. So there are no tests performed to analyse sensible data or any other kind of xss, sql injection or ocsi attacks.
So the conclusion is, that in your testcase our tool wasn't able to find any weakness, because is not part of the package.
I read your article and this was released couple of days back: defensecode.com/webscanner.php
I thought you might want to check it out.
I will for sure. Thanks @samirmeh !