DEV Community: James Cooper

Deploying Then Securing the OWASP Juice Shop, Part Six of ?

James Cooper — Sat, 06 Jan 2024 00:00:00 +0000

Penetration Testing: Amateur Hour

In this post, I am essentially going to fire up the OWASP Juice Shop (OJS) locally, navigate to the scoreboard to see the intended challenges, and then have a go at solving as many as I think I have a hope in heck of achieving. Given that I am not a penetration tester (in fact, I think I'd probably be rubbish as a professional pentester if I attempted it), I don't expect to solve all that many of the challenges, at least not without getting some significant hints from elsewhere. We'll see how I go, though.

This post is likely very much just going to be a long, rambling, lightly edited series of notes to myself as I go through trying to solve the challenges. Hopefully, that stream-of-consciousness style will actually be of help to someone, though. With any luck, I'll end up including the missing piece of the puzzle for someone else looking at one of the challenges, and enable them to go solve it themselves. Be warned, though: I'm not going to attempt to redact or hide things in this post, so there will probably be a bunch of spoilers littered throughout.

For reference, I ran OJS locally via Docker, using the Docker image tagged v15.0.0, with hash 178d89b55c46ae8b4167ddb28ce52ab4284ebce0766575c3943367ff09e27c97. By the time I started working on this blog post, a whole new major release had already come out, but I started the series with v15, so I'll stick with it. I activated said Docker image using the command docker run --rm -p 3000:3000 bkimminich/juice-shop:v15.0.0. I accessed the running OJS instance through my browser at http://localhost:3000.

If you're interested in having a go at the OJS yourself, you should definitely check out the companion guide available (as of the time of writing) at https://pwning.owasp-juice.shop/companion-guide/latest/index.html. It also includes solutions, though I imagine there are also plenty of other blog posts and the like out there which give much better explanations of how to solve challenges than I do.

Wait, what happened to parts three, four and five?

When I started on this blog series, I intended to spend a few posts exploring different ways to deploy the OJS to AWS. I did indeed do a bit of that in post two. It turns out, however, that this was both more complicated and fiddly than I had anticipated, and moreover that it is actually really interminably dull to me. Cloud deployments are, of course, a very important aspect of modern software development, but they're not something which excites me in the slightest. Rather, it's the application security aspects of all this that I'm interested in.

Given all this, and that I'm currently working at a job where the cloud deployment stuff is largely all handled for me (and thus I don't need to learn all about it right now), I have decided to skip trying to finish the deployment posts for the time being. I'm certainly not ruling out going back to it, but I don't anticipate working on that any time in the near future. Thus, this post. That still doesn't explain the lack of part five, though. For part five, I intended to model threats for the OJS, and then use that to help inform the later posts in this series. After all, a lot of people who know what they're talking about suggest that threat modelling is an excellent first step. E.g. that was the key message of the article "Coming of Age" by Stefano Zanero, that appeared in the September 2023 edition of the Communications of the ACM.

The problem is that I have somehow managed to go years in learning about AppSec while only ever developing the most rudimentary understanding of threat modelling. It'll take me some proper background learning to be able to produce anything worthwhile for this blog series, and I'm keen to get (re)started on it. Thus, I'm moving straight to fumbling around poking at a local instance, to see which of the challenges I can solve based on what I currently know (and the tools I have installed on my home computer).

Uh, Part Two?

Turns out that I never actually published Part Two to dev.to. I can't remember why not, but I imagine it's because it is rather heavy on screenshots, and the dev.to importer really doesn't handle them well. Actually, it doesn't really handle any part of my posts well (perhaps because I'm using Hugo as a static site generator), but I usually can just copy and paste the Markdown source into the dev.to editor, and it's all fine. Except, images require me to rewrite their embedding entirely, which was OK for this post with five images, but I can't be bothered doing it for that whole post. In the meantime, until I discover a better way of doing it or automating some sort of conversion, you can read Part Two on my personal site.

Image captioning

Which brings me to a related issue. I can't work out how the heck to use the standard dev.to Markdown code to embed an image to include a caption with it. It comes with a built-in part for the alt text (which is good), but the editor guide page just says that you can use the figcaption HTML tag to include captions, with zero explanation of how you combine that with the Markdown command.

On my personal site, I always include both captions and alt text, and I would like to replicate that here. If someone knows how to combine the two, I would appreciate you informing me. I could find multiple posts on dev.to discussing use of the figcaption tag, but nothing about using it in dev.to itself.

First Impressions

As I said in one of the earlier posts, I managed to find the score board myself. Honestly, there wasn't anything smart about it. I just guessed that the URL would probably be something like domain/scoreboard, and tried a variation or two on that. I probably should have taken the first piece of advice of the introductory tutorial, though, which said

Look through the client-side JavaScript in the Sources tab for clues. Or just start URL guessing. It's up to you!

It turns out that, most of the way through the main.js file, they list all the paths for the pages on the domain. The name of the relevant JS component is obfuscated, but, of course, they can't actually obfuscate the name of the path without breaking things.

The score board lists all sorts of challenges, with an estimated difficulty rating for each. For the time being, I'll focus only on challenges rated from 1-3, as higher challenges are either potentially beyond my skills and/or will take significantly more effort than I can be bothered with right now to achieve. I'll include all the categories, though, since I don't think there's any one particular thing I'm especially good at.

Besides the challenge of finding the scoreboard itself, a handful of other challenges jump out at me as ones I might be able to achieve. Especially some of the ones that sound like they probably are about bypassing client-side validation. Near the bottom, however, one jumps out at me as definitely a good place to start: "Security Policy". It's description reads

Behave like any "white-hat" should before getting into the action.

I'm not 100% sure what they're talking about, but it seems like a good idea to investigate this first. The mouseover tip says something about reading the security policy, so they're probably talking about making sure you understand what's considered in and out-of scope for the testing. I'll click on the extra hints button to find out more. That opens a link to what I think is probably the official walkthrough for the challenge. Unfortunately, it looks like they have re-jigged the URLs on the website since v15 of OJS was released, so it gives me a missing page message. Ah well, it's pretty easy to find that page again by just going back to the base domain and starting from there.

The write-up just has to this say

This challenge asks you to act like an ethical hacker. As one of the good guys, would you just start attacking an application without consent of the owner? You also might want to read the security policy or any bug bounty program that is in place.

So yeah, they were talking about the usual practice of getting explicit permission and defining scopes, etc. Of course, in this instance, the documentation around the OJS more or less has already given explicit permission (plus I'm only running the system locally in a Docker container on my own computer, so yes I can grant myself permission to attack that system). I'm dimly aware of the security.txt standard, so I'll give that a try. My first attempt, http://localhost:3000/#/.well-known/security.txt, doesn't seem to do anything. Maybe they don't have one? Trying it without the .well-known part doesn't seem to work either.

I move on to try to search through the code again, much as with the score-board. No luck there. I also crack open the privacy policy (getting that solution ticked off in the process), just in case it is linked in there, but no. At this point, I'm a bit stumped about all this. Ordinarily, I wouldn't nearly yet revert to checking the answers, but since this one is all about engaging in good practices up-front, I'll make an exception. The solution seems to say the same thing as what I tried, though. Except, I eventually notice that in my original attempt there was a # in there, but there isn't in the listed solution. Changing to http://localhost:3000/.well-known/security.txt does indeed solve the challenge. There's possibly some relevant info in there, so I'll keep that tab open for now.

That's three down, many more to go.

Some Basic Improper Input Validation

The below recounts the things I first tried (roughly in chronological order), but almost all of them were just uses of the same single method: Sending a legitimate request, then using the Firefox developer tools Edit and Resend functionality to modify some part of a web request to achieve something that you shouldn't be able to do. In general, this only works because the web application doesn't perform sufficient server side validation of the received inputs, instead (wrongly) trusting that the frontend validation takes care of it.

Zero Stars Given

Looking through the (filtered) list of challenges, some of the ones that catch my eye are from the 'Improper Input Validation' set. Maybe because I have a little experience of doing this sort of thing in the SANS Holiday Hack Challenge, I feel like I could make a stab at it. E.g., the challenge 'Zero Stars', which says "Give a devastating zero-star feedback to the store." My initial thought for this is that probably there is some sort of validation on the feedback form to prevent someone submitting a zero rating, but anybody who bypasses the GUI is unstopped. I.e., there's front-end validation, but no back-end validation. Turns out the front-end uses a slider to set a score between 1 and 5.

The easiest way I know of to try out fiddling with such things is to submit a valid request, and then use the browser's built-in development and debugging tools to replay the request with an updated payload. So, F12 it is, to crack open Firefox's dev tools. I fill out the feedback form with some dummy info and a 1-star rating, and watch the network requests. I notice that, besides the POST request to the /api/Feedbacks endpoint, there are also GET requests to whoami and /rest/captcha/. I presume the latter validates that the answer provided for the captcha challenge is correct, but I'm not sure what the other one is about. They could well be interesting later on, but I don't think I need them right now to achieve the challenge. I also notice that I get back as part of the response the ID 8. So there's probably seven other earlier feedback items in the database.

I do wonder at this point, though, whether I can do a GET request to the Feedbacks endpoint mentioned earlier. I try just requesting it via cURL, but get an error an invalid token. Presumably, I need to include some header or another for the endpoint to accept it. I'll try again via the dev tools, but first the zero-star rating (just in case I muck something up). I update the rating entry in the replayed request's body to 0 and hit send. Success! The ID on the response to this one is 9, suggesting that the feedback items are indeed stored with sequential IDs.

While I'm Here

Looking through the list of challenges for others that I can probably do while I'm here, I notice 'CAPTCHA Bypass', which is about submitting 10 or more instances of feedback within 20 seconds. Given that I could already replay the initial submission to do it once, if I can just repeat that same request in a for loop in a script, that should be trivial. I also discover that Firefox very helpfully actually provides an option to put the correct cURL command to repeat the request onto the clipboard. So, between that and a sliver of Powershell, that challenge is also already solved. The precise command used was:

1..10 | % { curl "http://localhost:3000/api/Feedbacks/" -X POST -H "User-Agent: Mozilla/5.0 (Windows NT 10.0; rv:121.0) Gecko/20100101 Firefox/121.0" -H "Accept: application/json, text/plain, */*" -H "Accept-Language: en-GB,en;q=0.5" -H "Accept-Encoding: gzip, deflate, br" -H "Referer: http://localhost:3000/" -H "Origin: http://localhost:3000" -H "DNT: 1" -H "Connection: keep-alive" -H "Cookie: language=en; welcomebanner_status=dismiss; continueCode=xmOK5wX8WKPpQY6Mr4neoLglBOAP9LHY7dNyjmbvxZEJq2D3V7a91zkR9EB7; code-fixes-component-format=LineByLine; continueCodeFindIt=2xaOBEyblGWQg2ormknJzea1vN6B4LXa4p503j7DLVERKxwq9XOdMYPJRvDb; continueCodeFixIt=O0Wmz4K5Ede4DGRP6WrJmw9kbB8XAvQoxyYLpOgzjq3QlN70VZonvM2e5wMn" -H "Sec-Fetch-Dest: empty" -H "Sec-Fetch-Mode: no-cors" -H "Sec-Fetch-Site: same-origin" -H "Content-Type: application/json" -H "Pragma: no-cache" -H "Cache-Control: no-cache" --data-raw "{""captchaId"":0,""captcha"":""-23"",""comment"":""ashgah (anonymous)"",""rating"":0}" }

That gives me two solved challenges with banners across the top of the scoreboard page. One of the other challenges says about closing more than one banner at once. Fortunately, I saw somewhere in the introductory sections of the "Pwning OWASP Juice Shop" companion guide that you can close more than one at once by holing the shift button when clicking. Turns out that wasn't a lie, and now I have another challenge solved.

Bullying a Chatbot

There's another challenge listed "Bully Chatbot", with the description

Receive a coupon code from the support chatbot.

It also is tagged with 'Brute Force' and 'Shenanigans'. This all leads me to suspect that if you can ask for a coupon code enough times in a sufficiently short space of time, you can get a coupon code out of it. Which means this challenge probably needs a similar approach to the last one. I take the exact same approach as with the 'CAPTCHA Bypass' challenge. I open up the chatbot section, write a message reading something like "coupon", and see what happens. The chatbot responds with something about how it can't give me a coupon code, but I now have a record of a web request asking for a coupon. I again copy the cURL command for it out of Powershell, wrap that in the Powershell shorthand for a for loop, bump the number of iterations up to 100, and hit enter. The confetti cannons go off, and apparently I have beat the challenge. It looks like I managed to beat the same coupon code out of it a few times, but have to scroll back up through the output of all 100 requests to find it.

GETting On With It

At this point, I remembered that I was wanting to check whether it's possible to access feedback with a GET request. So, I try the exact same request as before, except I change the HTTP verb from POST to GET, and in response I receive a list of all the feedback. The only thing that's of any interest to me in there is the fact that one of the comments seems to include some HTML. That might be how I could have a go at the "DOM XSS" challenge. Beyond copy-pasting the provided HTML into the payload, I also try changing the username from anonymous to Bjoern, and the rating to -1, just to see what happens. That pops the "Error Handling" challenge, but doesn't seem to do anything else, least of all tell me I managed the XSS. It kind of looks like they might have some input sanitisation going on that prevents it from being totally trivial. I will have to come back to this.

Repetitive Registration

By this point, I'm running out of challenges where I'm pretty sure I know from the start what to do. One does catch my eye, though, named "Repetitive Registration". The description of it simply says

Follow the DRY principle while registering a user.

I'm not totally sure what this means, but I can guess that it probably is about not repeating the exact same password when registering a new user. So, off to the new user registration section. A random username, a random password, and a slightly different version of that password later, and it's another challenge solved. Turns out they did indeed mean that you register a user without repeating the same password.

Empty User Registration

Oh, turns out there's another similar challenge to the last one. This time, register a user without even providing an email address or username. That same old 'edit and resend' trick to the rescue! This time, I just make the JSON for the username and password empty strings, e.g. { "Password": "" }. The confetti cannons go off, and it's another challenge done.

"View Basket"

By now, I'm running out of challenges I think I can solve through the one trick, but I notice one last one I might just be able to achieve. Viewing another user's basket. I just go back to the store's front page, put something into the shopping basket, navigate to the basket view page, and find the relevant HTTP request to get the contents of the basket. Then, I try sending that back but with a different user's ID. Turns out the OJS apparently doesn't validate for a basket view request that the requestor is actually authorised to view that basket (usually normal customers should only be authorised to view their own basket, and denied for everybody else's).

Moral of the Story

So by now I'm beginning to run out of ones that are totally obvious to me, and for which I don't think I need any further help. Almost everything I did here just revolved around modifying a legitimate web request and sending it again, to make the system use its normal functionality to do something a normal user isn't meant to achieve. The weaknesses exploited pretty much all relied on two failures in the secure development system (deliberate in the case of OJS, of course):

A lack of server-side validation of request parameters, presumably instead relying on client-side validation.
Assuming that people wouldn't observe actions the frontend undertakes by itself, and then imitating them in a way outside of that intended for non-malicious users.

This leads to two overarching morals of the story so far. Namely, that you should always assume that any web request the frontend makes can and will be observed – and abused – by users, and moreover that you should never rely on client-side validation to prevent invalid inputs. Client-side validation is used to help legitimate users interact with the system sensibly, but can always be bypassed by those with ill-intent. You must use server-side validation, away from the direct control of end-users, to confirm that parameters are valid and don't enable someone to do something they shouldn't.

OSINT

Thanks in large part to having taken part in the challenges for CHCon 2021, which focused pretty much entirely on OSINT, I think I might have a decent chance of completing some of the challenges under this category. Besides the general lesson of not using secrets that are actually on the web for anyone to find, there's likely not really anything especially instructive in completing the challenges. Thus, I'm just going to skip them in this blog post. I am awfully curious about the one for Bender, however, given that I have seen just about every Futurama episode at least once (if not many more times than that).

Administrivia

There are a few challenges that seem to relate to logging in as an admin user or accessing the admin section. Presumably, if you can log in as an admin user, you can access the admin section, so it seems like focusing on that is likely to be the more useful approach.

Admin Section

Two of the ones I can see involve logging in as the admin user in some fashion or another, but one of them just says to access the admin section of the website. Based on that, I'm guessing that the admin section isn't actually only available to admin users. Perhaps a link is only shown in the UI to administrative users, but anybody who guesses the correct link can find it. The mouseover text on the hint block for the challenge seems to back that up, so I'll try that.

I first try navigating to http://localhost:3000/#/admin, but that just gives me the front page. I then try http://localhost:3000/#/administration and find a 403 page. Looks like there actually is some access control on this page. I also try both without the hash, but that doesn't get me anywhere.

I realise that I'm not actually logged in as any user at this point, however. Possibly it's open to every registered user, but blocked simply for people who aren't signed in. So, off I go to sign in as a regular user. Except, that also doesn't work. I get the same result. It seems like you do need to be an admin user to access the page. At least I know where it is, though.

Which brings me to another point. Any unauthenticated user should always be shown 404 error messages for any page they are not authorised for. Otherwise, it becomes trivial to map out the real pages simply by requesting them and checking which ones return 404s vs 401s or 403s. Whether this should also apply to authenticated but unauthorised users is much more context dependent. If any random member of the public can create an account and thus be authenticated, you'll probably want to follow this approach. If you only have a very small number of users, all of whom are considered known and trustworthy, it's probably OK to give them a 403 message instead.

Anyway, it looks like I will need to log in as a user with admin access to manage this one, so let's try that next.

Admin Registration

This challenge has the description:

Register as a user with administrator privileges.

It sounds to me like there's probably some way to specify a user's assigned role when sending the input data for creating a new user. Perhaps the server accepts a role as input and blindly assigns that if present, rather than some server-side logic figuring out the correct role (which, for a customer-facing application will basically always be the standard user or customer role). Actually another one to try out the Edit & Resend system with, methinks.

I'll go through the user registration process again, but this time I'll watch the request and response to create a new user. This generates a POST request with the following request body: {"email":"***@example.com","password":"***","passwordRepeat":"***","securityQuestion":{"id":2,"question":"Mother's maiden name?","createdAt":"2024-01-05T20:36:38.850Z","updatedAt":"2024-01-05T20:36:38.850Z"},"securityAnswer":"***"} (bits replaced with *** just because, even though I used garbage values). The response body, however, has {"status":"success","data":{"username":"","role":"customer","deluxeToken":"","lastLoginIp":"0.0.0.0","profileImage":"/assets/public/images/uploads/default.svg","isActive":true,"id":22,"email":"***@example.com","updatedAt":"2024-01-05T20:59:21.429Z","createdAt":"2024-01-05T20:59:21.429Z","deletedAt":null}}. I notice the role field in there. Perhaps if I include that in my request body, too, with "admin", it might just create someone as an admin user?

I add "role":"admin" into the request body and resend it. That triggers the confetti cannons for this challenge, so apparently my new user has admin privileges. After logging in as that user, I can't see a link to any administration page in the UI, but I'll try out the link I guessed earlier. Yes, success! Instead of a 403 banner, I see a list of registered users, and the confetti cannons go off for the 'Admin Section' challenge. So, y'know, remember never to accept the role specified by a user in their registration request.

I note, though, that even after accessing the page, there never seems to a link to the admin section displayed in the UI. Possibly it was always intended that you had to access the page by knowing the link. Which, by itself, is a terrible idea. With it gated behind the admin role requirement, though, it's not so bad. After all, it should be very difficult to become an admin user illegitimately.

Cookie Monster

Looking at the cookies for the website, I see that there is one for the JWT token, which includes a listing for the user's role. Perhaps you could just change the cookie to grant yourself the admin role, and maybe the system will accept it from the cookie. I'm not sure if that would work, though, since it looked like the cookie was actually using signatures and validation, so updating it might cause the new JWT to be rejected. Perhaps it may be possible to change the validation algorithm used to the blank one (I think you just specify `"alg":"none"), but since the above worked, I didn't bother with trying that.

Now That We're In Charge...

Since we're in the admin section and can see a list of users, let's take a squiz at what info we have on them. Unfortunately, it seems that it's not a whole lot. We just get an email address and the date of registration and last update. Still, we now know the login email address for every registered user, which could well prove handy in future.

I also note that there is a display of the customer feedback for the OJS listed here, with trashcan icons next to each one. One of the other challenges, 'Five-Star Feedback', has the description "Get rid of all 5-star customer feedback." Assuming that there aren't any hidden feedback items, then it would seem like it's trivial to finish this challenge from the admin panel. In fact, there is only one item of five-star feedback listed currently. Hitting the trashcan icon next to that is sufficient to solve that challenge, too.

Login Admin & Password Strength

I tried out another couple of challenges around trying to log into the administrator's account (which we know from being able to see the list of emails uses the email address admin@juice-sh.op), but didn't manage to get anywhere. Thus, given that they both have associated tutorials, they seem like good candidates for the next section of this post.

Get Help

By now I have run out of ones that I can figure out without any sort of help (and which don't sound like they'd be a whole load of work). There are some I think I'm pretty close on, however, like the basic cross-site scripting ones or the one about finding an exposed metrics system's endpoint. Furthermore, there are a few remaining challenges that apparently come with walkthroughs or tutorials, so I might as well work through those while I'm at it.

XSS

The first thing I think I'll look at is revisiting the XSS challenge that I couldn't quite figure out before. I don't want to go straight to the guided tutorial, so I'll look at the extra hints they provide on the website, and see if I can get far enough with just those. The hint there basically just tells you to go look at the next hint, which is for one of the challenges that aren't actually available by default when running in Docker. Basically, it just says to look for a part of a page where the user's input is incorporated into the page.

I went through just about every page in the whole shop, trying out the instructed HTML. No luck with any of them. Either they didn't have anywhere for user input, or the user input was either scrubbed appropriately or not actually displayed back. I'm just about ready to give up and run the tutorial, when I realise that there is a website search function, and that I haven't actually tried with that yet. I simply copy-pasted the sample command into the search box and hit enter, and the challenge is completed. Yip, it's that simple, when you know how to do it (in this instance).

Bonus

Somewhere else on the scoreboard under "Bonus Payload", they list another, larger, input to try. It's the exact same process to solve this one, too. Except now instead of a little pop-up message, it links you to a recording the OJS jingle. I suspect that this challenge was included more to give them a way to make people listen to the jingle, rather than for some educational benefit.

A Modern Prometheus

The "Exposed Metrics" challenge says to find an endpoint serving up usage data for a popular monitoring system, where said system is clearly indicated to be Prometheus. I would guess that probably the endpoint in question is exposed on the default path and port, and without any auth requirements put in front of it. Depending on what's available through it, this may or may not be a bad thing. You should definitely default to keeping things buttoned-up, though, unless you can be really sure that there's nothing sensitive in there. You never know what a sneaky attacker can find and exploit.

Anyway, the point is that this challenge is probably trivial if you bother to read some documentation. In fact, most likely there will be some sort of 'getting started' document that describes the default endpoint to query. The page labelled "First Steps" seems like a good place to look. Yes, here we go, that page includes the sentence

Prometheus expects metrics to be available on targets on a path of /metrics

That seems like a good first thing to try. In this instance, it's also the last thing to try, as simply pointing one's browser to http://localhost:3000/metrics is sufficient to solve this challenge. Turns out I didn't really need extra help on this one. It highlights an excellent point, though, namely that you should just do the first part of the beginners' tutorial and then stop and call your production system working. That's good for initially experimenting with and learning a new system, but it's pretty much never appropriate for actually deploying it anywhere besides your development machine.

Return of the Administrator

For the two administrator-account-related challenges I couldn't complete above, I'll first check the hints, both in the OJS itself and in the Pwning OJS guide.

Login Admin

To be honest, the hints don't really give me much help with this. I had already figured out that I probably want to have a go at the login form, but hadn't really got further than that. One hint mentions trying to reverse the password hash, but I have no idea where to find the password hashes. Otherwise, it's not really clear to me what they intend. I'll have to resort to going through the tutorial.

Ah, turns out they're intending for you to use SQL injection. To be honest, to this day I have still never really quite got the hang of SQL injection (perhaps because I'm still not all that great at writing SQL), so I probably did need the tutorial for that. As it turns out, the magic string for this is something like ' OR true--, which you slap into the email address. As I understand it, the opening quote mark closes off whatever SQL query the string is put into, then the OR true ensures that the whole thing returns true (presumably it's assumed to be something checking that the password entered matches the password on record). The dashes are a comment in SQLite flavoured SQL, so that anything else in the query is discarded, rather than being kept and causing syntax errors when things don't fit right.

Anyway, following the tutorial gave me some idea of what they expected, and how to do it. Definitely worth resorting to the tutorial when all else fails.

Password Strength

The hints suggest that it should be fairly easy to guess the admin account's password. I tried earlier, though, using everything I could think of like "password", "letmein", "12345", "monkey" (I hear that is a surprisingly common password), "admin", etc., and combinations of the above, with and without a trailing "!". None of them worked. Again, the hints suggest using the password hash, but I still have no idea where that is supposed to come from, so that won't work. They also suggest attempting to brute force it using a password list and a script, which is a good idea, but I'm just too lazy right now. I shall instead resort to the tutorial yet again, and see what they have to say.

Hmmm, the tutorial doesn't really tell you too much, it basically just suggests to guess, then kinda lets you know when you're in the right place. Anyway, it turns out the admin password is simply admin123. I suppose I must have tried admin12345, but never though to try it out without the last two digits. The lessons here being things like never using such an obvious and easily guessable password,¹ that manually trying out some guesses can get you a long way, and that manually trying out guesses can be error-prone and scripting something to brute force it might well be easier in the long run. Assuming that people are foolish enough to use a password that can easily be gathered from such a list.

Help Unwanted

Ok, turns out that I didn't really need help for some of those challenges. If I had kept going with others (e.g. those with more than two stars) I almost certainly would have required it, though. At this point, however, I'm getting bored with writing this blog post (not with attempting to hack the OJS, though), so I'm gonna call it quits here.

Summary

So there you have it. A reasonable number of challenges solved, mostly just by tampering with the body of legitimate GET or POST requests and resending them. In every one of those cases, the OJS is too trusting or accepting of user-supplied inputs, and performs insufficient server-side validation. Perhaps in some cases it inappropriately relies on client-side validation to stop people submitting iffy values, which is a definite mistake. Meanwhile, other challenges can be solved by a little (vaguely-educated) guesswork, or reading a little documentation. The main morals of this story might be:

Never trust user inputs.
Always perform server-side validation of user inputs, rather than relying on your frontend UI to stop miscreants from submitting invalid stuff.
Don't leave anything sensitive exposed without further authorisation requirements, especially if that is available in a default or easily guessable location.
Be cautious about what can happen if users automate making requests. Strongly consider putting rate limiting on any API endpoint where a legitimate user is unlikely to need to use it at extremely high rates.

Of course, there are still myriad more challenges in the OJS to complete, many of which will likely require other techniques to solve, and other preventative measures to defend against. Thus, the list above is almost certainly thoroughly incomplete. Cybersecurity: It's a deep topic :)

If you think this post seemed interesting, I fully encourage you to have a go yourself. Give it your best shot without using any hints, but once you have spent a decent amount of time on something and haven't gotten any further, there's nothing wrong with looking for some help. After all, if you're giving it a go, you're probably not claiming to be an expert already, and sometimes the only way to learn is to get some help from elsewhere. Definitely some of the techniques I have seen used are things that I would likely never come up with myself, but once I have experience with them, perhaps by working through a walkthrough myself, I might know to try them out.

If you'd like to learn more about hacking websites/web applications, the Portswigger Web Security Academy is generally extremely well-reputed for a free resource. I believe it's somewhat focused on Portswigger's own product, Burp Suite, but that's just a (widely-used by professionals) tool to make doing a lot of these things easier, rather than something magical which enables people to do something they couldn't otherwise achieve. You can also have a go at the most recent few year's SANS Holiday Hack Challenges.

For paid training resources, I understand that Pentester Academy is pretty well-regarded, too. There are also things like Hack The Box and Try Hack Me. OWASP members receive complementary membership to an OWASP instance of SecureFlag. I don't know exactly how well-regarded each of them is, but I don't think any of them have particularly bad reputations. There are undoubtedly a vast panoply more resources, both free and paid, out there for web application security, but some of the above should be a good place to get started I imagine.

Lastly, if you think this stuff seems pretty neat, and you might like to do it professionally, you could look at becoming a penetration tester. It's not the right path for me, but it might be for you. It generally pays pretty well, and if they get bored with pentesting, pentesters generally seem to be able to go on to high-flying jobs in the cyber defence side of things. Having not done it myself, I can't speak too much to how to go about trying to get in, so I'll point you to Simon Howard's excellent resource on the topic: Getting Started as a Penetration Tester in NZ (2023 Edition). Mr Howard is very well respected in the New Zealand security industry, and can be considered reasonably authoritative on the matter. The post is New Zealand-focused, but I imagine a huge amount of the information applies in most countries around the world. By the time you read this, he may well have written a later edition, so it might be worth seeing if you can find that one.

Oh, and last of all but most importantly: DO NOT COMMIT CRIMES. Use your newfound hacking powers for good, and always get permission (preferably explicit written permission) from the owners & administrators of any system you target, before you take any action against it. Seriously, the difference between criminal acts and a paying job can sometimes be as simple as whether you asked first. If people say no, then move on. There are plenty of targets out there already for you to practice with.

Ideally, you would use a password manager to create and store long, random passwords for you. Or if you need to be able to memorise it, get the password manager to generate a good, actually random passphrase for you (all good managers come with such a generator these days). ↩

My Problem With Design Patterns

James Cooper — Sun, 17 Dec 2023 03:19:58 +0000

I Have A Problem With Design Patterns

Just in case the title and heading didn't tip you off, I have a problem with software design patterns. The problem isn't actually with the design patterns themselves, nor (necessarily) with the people who promote them. My problem, at it's root, is how people treat them with an unquestioning semi-religious idolatry, and also use them as a way to declare themselves superior to others in a most idiotic fashion.

Not Alexander's Pattern Languages

Before I go any further, I just want to point out (in case there's any confusion), I am not talking about the design patterns as discussed by Christoper Alexander in his book 'A Pattern Language'. At SPLASH 2022, there was an interesting keynote talk from a Japanese professor (whose name I have completely forgotten now...) discussing that field. At the start of his talk, he asked who in the audience knew about design patterns. I put my hand up, mistakenly thinking he was discussing software design patterns. No, turns out he was discussing something older, and probably more meaningful. Anyway, the point is that I'm vaguely aware of this other use of the term, but I am not talking about it here. Rather, I'm talking about design patterns as usually known in software development, with names such as the Visitor, Mediator or Observer pattern.

The Ferengi 'Bible'

The Ferengi are an alien race in the Star Trek universe. They were first introduced really weirdly (seriously, they were like these little barbarian dudes in loincloths with electro-whips—the first series or two of Star Trek: The Next Generation was frequently bad and/or weird), but very quickly morphed into a group focused primarily on their utter obsession with gaining profit. Star Trek: TNG largely ignored them after that, but they were explored in great detail in Star Trek: Deep Space Nine. In DS9, they are shown to approach this obsession with wealth accumulation a religious fervour, and central to this is a holy book known as the Rules of Acquisition, which is kinda like a cross between The Art of War, Proverbs from The Bible, and any number of books published in the 1980s promoting ruthless capitalism.

The point is, the main Ferengi character in DS9, Quark, eventually somehow gets to meet one of the authors of the Rules of Acquisition (I forget exactly how it works, given that the book was supposed to have been first published a long time ago and all the authors were long dead). The authors of the book are more-or-less seen as religious prophets, communicating the great holy truths of the universe to the Ferengi. Which is a great surprise to this author to whom Quark speaks. The authors merely tried to publish a book that would sell well, and named it "Rules of Acquisition" because a title like "Suggestions for Acquisition" probably wouldn't be as successful. They never anticipated that Ferengi culture would be so strongly shaped by their book, and, if I recall correctly, the author with whom Quark talks isn't terribly impressed by how the Ferengi have chosen to throw away some of their capacity for thinking for themselves and instead slavishly following the book.

I say all this because I have the impression that something similar has gone on with design patterns, starting with the so-called 'Gang of Four' book.¹

The Gang of Four Book

I'm pretty sure software design patterns have existed, in some fashion, since shortly after people started programming computers. They would have found that some approaches worked better than others, and thus continued using the better ones. Over time, the programmers would likely have developed a sense for the overarching concepts involved, and probably communicated that to the newer programmers coming on board. I don't know when such things first became formalised, nor when they first were referred to as "design patterns". Certainly they were vaulted well into popularity by the book "Design Patterns: Elements of Reusable Object-Oriented Software"—commonly known as the Gang of Four book.²

I have to admit, I have never actually got around to reading the GoF book, though I have certainly come across many references to it. So far as I can tell, it includes quite a lot of good advice for people using C++ around that time, and indeed considerable parts of it are informative for others working in an object-oriented language. I'm sure one could find many small criticisms of it if desired,³ but it seems like it genuinely was a pretty good programming book, and one where a lot of the advice continued to be useful years into the future (most programming books are outdated by the time they even are officially published, and ancient history after about three years).

The bulk of the book focuses on 23 design patterns that the authors had discovered/figured out with experience, and kept re-using as a good way to keep the software they were developing from becoming unmanageable. Basically, each design pattern described an architectural approach (for a wide variety of potential meanings of 'architectural'), and the sorts of scenarios where they had found it was useful. The benefits tended to be software that was easier to develop and maintain, with relatively low costs in other regards. My impression is essentially that was a compendium of programming/software construction techniques the authors had found useful, and intended to be the sort of book you keep around and refer to for inspiration when trying to work out how to structure a new piece of the puzzle.

If that was where we stopped, everything would be fine. As I said before, I'm under the impression that there really is some good advice in there.

The Problem

I said this in the introduction, but I'll repeat here: My problem with design patterns is the quasi-religious way people approach them, as if they're the patterns are great fundamental truths of the software development universe, brought to us by the prophets the Gang of Four in their holy book. I have genuinely see people talk as if they think that every single aspect of programming fits into one of the design patterns, and it's all simply a matter of determining which one is right for the situation at hand. I have also seen people using knowledge of design patterns as a form of gatekeeping. Those who are ignorant of them aren't 'real' software developers and are to be pitied. Programmers who are away of design patterns (and, usually, this specifically means those from the GoF book) but choose not to use them are heretics to be avoided or ridiculed—either way, they're clearly misguided and inferior.

Dijkstra is supposed to have said something like⁴

Please don't fall into the trap of believing that I am terribly dogmatic about [the go to statement]. I have the uncomfortable feeling that others are making a religion out of it, as if the conceptual problems of programming could be solved by a simple trick, by a simple form of coding discipline!

He was, of course, referring to this famous opposition to the use of goto statements in typical programming, because they tended to lead to far greater problems at the large-scale than they solved at the small-scale. Design patterns seem to me to be a similar idea, in that they're suggestions of ways to structure software to make it easier to handle in the large-scale, yet others seem to have made a wacky religion out of them. I have a suspicion that if you take the Dijkstra quote and replace "the go to statement" with "design patterns", the Gang of Four authors might well feel similarly. Or indeed, if they spoke to some of the zealots of the Cult of Design Patterns, they might have a similar reaction to the Rules of Acquisition author when he met Quark.

So, overall, my problem isn't with design patterns, but rather the way that people choose to approach them and abstain from thinking for themselves. I am yet to see any set of patterns or indeed any tool that can entirely obviate the need for one's own brainpower when developing software. The real best developers know when to use the typical approach, and when and why to deviate from it.

Not to be confused with other groups labelled the Gang of Four, such as (I believe) four prominent members of the Chinese Communist Party in the late 1970s and/or early 1980s. ↩
E. Gamma, R. Helm, R. Johnson, and J. Vlissides, Design patterns: Elements of Reusable Object-Oriented Software. Pearson Education, 1994. ↩
People certainly found things to criticise. E.g., the suggestion that many of the patterns in the book were really just crutches to work around limitations in C++ as it was at the time, and that these patterns were wholly unneeded in other languages. I don't know how well justified said criticisms were, but it is true that C++ back in the day was a much different beast to the modern era. ↩
Taken from https://en.wikiquote.org/wiki/Edsger_W._Dijkstra#1970s, accessed on 16 December 2023. ↩

Phrases in Programming That Irk Me

James Cooper — Sat, 28 Oct 2023 00:00:00 +0000

Irksome Lingo

It is common amongst the non-executive types to deride so-called 'execu-speak'. That is, words and phrases which sound trite, stupid or (sometimes) like disingenuous euphemisms. While there can sometimes be some justification for such criticism, quite a lot of that vocabulary is simply, in essence, the jargon of that field. Software developers are actually at least as equally guilty of overusing sayings, re-using lexicon from somewhere else such that it makes little sense in the original context,¹ and just plain using words and phrases that irritate me. I present below an incomplete list of said words and phrases in alphabetical order, with a brief description of why each one irritates me. I imagine that you find some of them perfectly fine or useful terms of art, and probably some things I say would irritate you.

Also, I'm likely to keep updating this over time, so you if you like some of this, you might be interested in coming back periodically.

B

Battle-tested

This is just so darn overblown and self-aggrandizing. There's an extremely high likelihood that your code won't actually be used in a battle, so why are you talking about it going to war? If you really were developing software which could be used in a real-life battle, the idea of programming in Ada wouldn't sound weird and foreign to you. You likely wouldn't be programming to run in the cloud however, much more likely you'd be developing for an embedded subsystem of some military vehicle. So if it's incredibly unlikely people will die as a direct consequence of bugs in your code, stop describing it in military terms, you tosspots.

Those people who develop software for military purposes and have seen it come out the other side smelling of rainbows really can use the phrase "battle-tested" because it's true. All the rest of you: find something less insulting to people on the front lines of the world's conflicts.

R

Rich

Everything seems to be "rich" lately. Type systems, core libraries, user interfaces, the small group of people that tech firms actually give a hoot about. In some circumstances, using the word does somewhat make sense, but it's another one that has been overused to the point of meaninglessness. Much like when you keep repeating the same word over and over until it just sounds weird (I think that is referred to as 'semantic satiation').

S

Single Pane of Glass

I get why this one was originally coined. It was possibly meant to refer to the actual panes of glass used in old CRT monitors, or just the idea that you have one window onto a situation through which you looked for everything. It's just that this one has been used ad nauseam, so it bugs me.

Story

I remember reading a rant many years ago from a then-software developer complaining that absolutely everything was being described as a "solution", even when that made basically no sense. I vaguely recall the phrase "a paperclip is not a solution." The word "story" seems to have replaced "solution" for this purpose. Everything now either is a story, or has a story. "What's the story around 'X' concept in 'Y' programming language?" "Does 'P' have a good story for 'Q'?" No, it reads and sounds like Rushdie's 'The Satanic Verses'—incoherent and incredibly dull.² Pretty much every time I hear someone use story in this sort of fashion it bugs me intensely.

I think this is one of those misused words that came straight out of Agile (or one of the things it kinda sorta amalgamated, or one of the things that latched onto it) with user stories. Which, while I find horrendously overused in some ways (not every single task description needs some inane, tortuous, imagined novella from the perspective of a nonsensical character), does in and of itself make sense. Somehow, though, the use of the word story leaked out and spread its smelly, greasy oil slick over our pristine beach, but celebrities have cleaned all the cute animals in media-friendly photo opportunities, and I'm left metaphorically scrubbing with only a toothbrush mucky rock after mucky rock of people misuing the word "story".

Honestly, this is probably the one which bugs me the most of anything, because ALMOST ALWAYS WHEN SOMEONE SAYS "STORY" IN SOFTWARE DEVELOPMENT THESE DAYS IT MAKES NO FREAKING SENSE.

Some Things That You Might Think Would Bug Me, But Actually Don't

B

Bikeshedding

Ok, this one does kinda irritate me because I hate people turning other types of words into verbs like that. I would prefer something like "discussing the colour of the bike sheds to death" instead, but that's rather more of a mouthful, so I can appreciate why it ended up that way. The idea behind the phrase is good, though, because it does communicate well something that happens in meetings and discussions the world over. Namely, that people really do have a tendency to fixate on trivial details and speculate on irrelevant matters when they should be focusing on much weightier, but less straightforward, issues. I don't have any better way to describe the idea than this, yet I observe it frequently in practice.

Y

Yak Shaving

This phrase actually sounds pretty stupid, and doesn't really give you any concept of its meaning when you first read it. It actually does (kinda) make sense, however – sitting there trying to shave a yak does seem like it would be awfully tedious and maybe feel a bit pointless – and it covers something that there isn't really another phrase for. Plus, it gets a bit of a free pass from me since it's a Ren & Stimpy reference.

So very many terms that were seemingly first used in Agile have fallen victim to this. Perhaps, because like almost everything that was labelled Agile (but usually specifically meant Scrum), people got so indoctrinated that they don't understand there's other ways of saying and doing things. ↩
Yes, I really have read 'The Satanic Verses' by Salman Rushdie. I wouldn't recommend it. It is incredibly obtuse and boring. I wonder that muslims weren't so stirred up by it because it's such a waste of one's time, rather than because of the thinly-veiled blasphemy. If you're thinking about reading it, go read the dictionary instead. It's no more dull, can be shorter depending on which version you have, and at least you'll learn something. ↩

I'm a little iffy on Passkeys

James Cooper — Sun, 22 Oct 2023 00:00:00 +0000

I'm a little iffy on Passkeys

In case you haven't heard, passkeys are the new saviour of the security world (yes, I do say that with a tinge of sarcasm). In fact, Google apparently just recently switched their default credential system for Gmail over to passkeys from regular-old usernames & passwords. Passkeys are so strongly considered to be the way of the future that both 1Password and BitWarden seemingly bought passkeys-focused startups so that they could add the capability to their products (I didn't manage to track down any announcements or old news articles confirming that, though). I assume all the others in the password manager game will also try to do the same.

Just the other day, when I had 1Password open at work, and it happened to be on the entry for my work GitHub account, it was telling me to go set up a passkey and store it in 1Password. Given that passkeys are supposed to be specific to a given device, though, I am forced to wonder what the point of keeping it in a cloud-syncing system like 1Password is actually supposed to be. Moreover, why can't I simply rely on the operating system's in-built support? (This was on my work-issued device, which is still on Windows 10, so maybe there isn't such great support for passkeys there.)

As I understand, passkeys basically introduce device-hosted public-private key systems, for just about every device with some sort of hardware support for cryptographic procedures. It seems that when setting up a new passkey, your device generates a public and private key and passes the public key to the remote authentication system. At login time, the authenticating system uses that public key to issue a challenge, which can only be solved by someone possessing the private key. I may have the exact workings of this wrong; I'm no expert. I seem to recall having heard somewhere that passkeys are just an implementation of the same standard used for hardware MFA keys, but I'm uncertain how true that is.

The Good Bits

There is indeed a lot to like about passkeys. For one thing, they're supposed to be highly phishing-resistant.¹ In theory, they also can be managed by the OS transparently to the user. As in, the user never sees anything happen or even needs to remember anything, but their device authenticates to the remote service automatically. They're also random and should be cryptographically secure. Thus, it should be pretty much impossible for anybody merely to do some sort of brute force attack and get anywhere. Dictionary attacks and password spraying would become nearly impotent. I presume rainbow tables would also no longer be of any use since the number of solutions that would need to be precomputed would be mind-bogglingly huge, and even then, there is likely an enormous list of possibilities that would need to be tested via brute-force since there's an extra unknown secret that complicates things. Kind of like password salting, but perhaps on a vastly greater level of effectiveness.

So, all up, they should be more secure than many passwords and substantially more convenient for users in the case that everything is going well. To the extent that users don't necessarily even notice an authentication process occurring, but it all happens appropriately in the background, nevertheless.

The Bad Bits

First and foremost, my main issue with passkeys is that the people pushing for them seem to assume that everybody always has an up-to-date, powered smartphone with them at all times. That will often be true for many in relatively wealthy countries, but it is not guaranteed even there. There will still be plenty of people who can't afford them or want to choose not to use a smartphone for whatever reason. Moreover, people lose their phones every day, never mind that batteries can run flat. And what if, for whatever reason, you want to log into an account of yours on a device not owned and controlled by you when you don't happen to have on hand one of your devices with access already set up?

I also don't see that they're nearly as needed as ten or so years ago. The whole thing about getting rid of passwords basically assumes that everyone still uses the same low-quality passwords everywhere. While that's probably true for some, plenty don't do that now. Password managers present another good solution to the problem. Heck, just dreaming up passphrases and writing them down in a notebook can avoid many of the same weaknesses that passkeys try to avert. Plus, I have the impression it's much easier to reset a password than a passkey.

Another thing I'm a bit uncertain about is how passkeys appear to reduce multi-factor authentication down to single-factor. Basically, if you have control of a device, then it seems to me that you have control of the credentials, whereas, with a password, someone still has to remember something. If possession is equivalent to authentication, then so-called "rubber-hose cryptography" would seem to become more effective, not less. I mean, when it's a password, you need the victim to remain conscious. You don't need them awake or even alive if the digital key is merely something in their possession. In theory, this is stopped by most devices requiring some approval process on the device itself, but if all that is required is a fingerprint scan (which seems to be the usual thing mentioned, along with facial recognition), then I'm unconvinced you need to keep someone conscious to exploit their device.

In Conclusion

I could write more on this topic, but I'm too lazy right now to bother to track down the relevant sources. Instead, I'm just writing this from my memory of reading news stories about passkeys over the past year, so take everything with a grain of salt.

When you get down to it, I suppose my concerns with passkeys aren't with passkeys themselves so much as taking password systems away and forcing people to try to use passkeys only. While passwords certainly aren't perfect, they're pretty well-understood, we have a solution to doing them relatively well (password managers), and they have one enormous advantage over passkeys in that people can memorise them. Sometimes, knowing a secret is exactly what we want.

I'm also iffy about how passkeys (or perhaps more, their champions) seem to expect a certain level of wealth and technical sophistication on the part of users. Just about everyone can get their head around a passphrase, and in certain circumstances, it can be a very good thing that those can be given to someone else. Neither of these seem to be true necessarily for passkeys. In their rush to make things more convenient for some, it may be the case that the big tech firms are further widening the digital divide. Not that I actually expect any of them to care. They've made it plainly obvious many times over already that they don't.

The other month, I was talking to Yuriy Ackermann, who has long had a lot to do with the FIDO Alliance and passkeys. He told me that some of my concerns here might not be as significant as I suspect and that the passkeys standard isn't as restricted to smartphones as it appears. Instead, he suggested, the information security media have essentially just repeated the marketing talk from the big tech firms such as Google and Apple, who have been pretty focused on their respective smartphone systems. I don't know if that's true, but Yuriy is (to the best of my knowledge and understanding) an expert in this area, so I'm inclined to listen carefully to him.

I guess we'll have to wait and see how things shake out. Hopefully, my scepticism proves unfounded. I imagine there are going to end up being a few people locked out of their Google accounts, however, when something goes pear-shaped with the passkey(s) for their account and they have no way to reset things—after all, Google is notorious for not having any functional customer support from real people.

I'm not totally clear on how that is the case, but I presume it is because the secret – the device's private key – is never transmitted elsewhere, thus meaning that the phishers only get a valid solution to their specific challenge, not something they can re-use anywhere else. ↩

On Government Surveillance Via Data Brokers

James Cooper — Sun, 10 Sep 2023 00:00:00 +0000

On Government Surveillance Via Data Brokers

I listen to a number of podcasts that have at least a partial focus on digital privacy. Most notably, these include The Privacy, Security and OSINT Show and Surveillance Report, while Risky Business also tends to touch on such matters at times. I have also listened to others in the past, but they either have stopped running, or I found something about them off-putting. I also read the Firewalls Don't Stop Dragons newsletter. There's plenty more out there, too. The point isn't really whose stuff I consume, but more that I pay attention to the world of digital privacy.

One issue that seems to keep cropping up in recent times is that of governmental bodies purchasing information from data brokers, apparently in lieu of obtaining warrants to obtain the same information. I'm not sure exactly how well substantiated those claims are, but for the sake of current argument I'll accept that this indeed has been going on. I have a feeling that there have been reports in the media that various high-level figures have publicly stated this practice has indeed occurred. The general thinking is that the law enforcement or intelligence agency personnel would likely have found it difficult to justify adequately the need for a warrant, and so instead they have resorted to an alternative method available to them.

Every time such a news article comes out, there are typically comments along the lines of how terrible it is that law enforcement and intelligence agencies are circumventing restrictions on such measures—usually with mentions of the word "constitutional", since these articles are usually in reference to the US, and people perceive the purchase of such data as violating the Fourth Amendment to the US Constitution.¹ These comments aren't unjustified, since it does indeed appear that the authorities are circumventing legal restrictions, but they inevitably are silent on a related point: Why should the authorities be prevented from doing something that any random private individual with sufficient money can do, i.e., purchase information from data brokers?

To the best of my knowledge, there is never any suggestion that anybody else cannot go to these data brokers and make the same purchases, assuming that they have deep-enough pockets. Therein lies the real issue, to my mind: these data are collected and collated by data brokers, and made available to whoever can stomach the asking price. Whereas, it seems to me that the purpose of warrants is to enable authorities to exercise powers unavailable to the general public. To the best of my knowledge, there is no legal right for any random person on the street to enter someone's dwelling and search their property, and thus the relevant governmental bodies are required to go through the warrant process because they're exercising a power beyond normal rights. The trade-off to having those powers at all is the requirement that they are overseen through a (theoretically) independent review process. If I'm not mistaken, more than one person who was obviously guilty has gone free because the damning evidence against them was barred from court because it was technically seized without an appropriate warrant.²

It seems to me that, really, the issue at the heart of this matter is that the information is made available to anyone. In fact, the relevant governmental bodies are probably the last group I would want to see restricted from being able to use the data. That's not to say that I necessarily want them to be able to use it, either, but at least they are supposedly on our side (on paper, at least), and serving and protecting us. Any random private citizen, or indeed any number of morally-dubious businesses, still have access to that same information but vastly less prima facie responsibility to use it for purposes that won't cause harm to others. Whereas the law enforcement and intelligence bodies, at a minimum, pay lip service to doing things for the good of normal people, and are theoretically accountable to elected officials.³

Basically, what I'm saying is this: The real issue seems to me to be that these data are available to anyone with enough money, rather than that the government might use them. Let's get access to them restricted for other people, and then we can talk about whether governmental bodies (i.e., the only ones involved who are definitely on our side, at least on paper) should be able to get their hands on them. Or, better yet, how about we see to it that nobody gathers the data in the first place, and then the whole issue goes away entirely.

I'm unconvinced that the Fourth Amendment truly would prohibit such purchases, but as I am not well-versed in US constitutional law I won't further debate the matter here. ↩
Of course, there are certain other circumstances where the authorities can perform searches without first obtaining a warrant, but presumably in these "innocent-on-a-technicality" cases, those alternatives are not considered justified. ↩
There seems to be a perception amongst some parts of the general populace that the people who work for just about any governmental body sit around dreaming up new ways to oppress people (and waste money). I can confidently assure you, based on my experience working at Auckland Council, that this is most definitely not true about a great many people who work for some arms of the government. Not only do the vast majority of the people I came across have no interest in oppression, most of them genuinely want to do things to benefit the public. ↩

My Weekend With PHP

James Cooper — Sun, 30 Jul 2023 00:00:00 +0000

My Weekend With PHP

For certain reasons, I spent most of this weekend ~~diving into~~ scratching the surface of PHP, a language of which I have been aware for a long time but never actually touched until now. In case you somehow are reading this yet have never heard of PHP, it summarises itself thusly:

A popular general-purpose scripting language that is especially suited to web development.
Fast, flexible and pragmatic, PHP powers everything from your blog to the most popular websites in the world.

For reference, I was using PHP 8.2, specifically. That being the most recent official release available at the time (I'm pretty sure 8.3 is already in pre-release, but I didn't need or really want to dive into the cutting edge). I have the impression that there probably have been a number of changes from 8.0 onwards that have made it a much better language to my tastes, but I can't say for sure.

I spent most of my time on Exercism's PHP track, working through the concepts curriculum there.

In the end, my impression of the language is mixed. Which, considering I had rather low expectations going in, is actually a net win for PHP.

Preconceptions

To be honest, I didn't have a great image of PHP in my head before this. To me, it mostly meant giant messes of spaghetti code that started off as something that somebody cobbled together in a few hours, and which had grown out of control since. Or, even worse, the thing that phishing crews used to operate their kits without giving all their secrets away in client-side Javascript. And generally, those phishing kits are Frankenstein's monsters of garbage code pilfered from everywhere and anywhere, then beaten into shape just enough to do what they want (i.e., steal people's personal information). It also reminds me, negatively, of websites from the 1990s. Oh, and complaints from my erstwhile colleagues at Cosive about MISP when working on CloudMISP.¹

What little I did know of it was that (at least, so I thought) it is a dynamically-typed language, with a pretty basic type system.² Furthermore, it is interpreted rather than compiled. Those aspects combined are just about some of my least favourite for a programming language, so I wasn't expecting great things. And, I had the impression it was pretty heavily imperative and/or object-oriented, which aren't inherently bad, but those who have spent more than five seconds talking to me about programming languages know that I favour the functional-esque ones. Or the more declarative ones, at least. So yeah, I wasn't expecting great things.

Running PHP and PHPUnit on Windows 11

Using PHP on Windows 11 is not trivial, but it's also not too hard if you're willing to work with WSL or some form of virtualization such as Docker or Vagrant/virtual machines. There are official PHP Docker images which let you run PHP scripts pretty easily. Sadly, as soon as you want to use PHPUnit, which I understand to be the dominant unit testing framework in PHP these days, things get much more difficult.

PHPUnit is not distributed in the official PHP Docker image. There are two main ways to install it into a given project. Using PHP's Composer dependency manager (think roughly .NET's NuGet) or by using a PHAR (think loosely the same as a Java JAR). Composer seems to be the generally-recommended option, but it turns out that Composer itself isn't included in the official PHP Docker image. In one way, this makes sense since it's a system developed by people outside the PHP developer team itself, but in another way it seems a bit silly since it appears to be the de facto standard these days for managing dependencies. Moreover, it turns out that installing Composer seems to be inevitably a long and convoluted process.

Sadly, the problems didn't stop once I got Composer installed in a Docker container. In theory, all that is required at that point is to use composer require phpunit/phpunit, and away Composer will go and install it for you. Which it did, but for some reason no matter what I tried, I could never get the darn thing to run successfully on the tests provided in the Exercism exercises. I eventually gave up on trying to do it myself at the command line, and turned to IntelliJ IDEA, since I figured with the PHP plugin it'd make configuring things much easier. That proved to be a vain hope, however. After spending probably about two more hours fiddling about trying to figure out why I couldn't get things working, I gave up again and installed the PHP-specific PHPStorm IDE (which, in theory, is just a stripped down version of IntelliJ).

Even then, the difficulties weren't over. Long story, short, I eventually figured out (with some assistance from the PHPStorm documentation online, which is actually somewhat decent) how to get it configured to use the PHP installation inside a Docker container, and then configure it to use PHPUnit. The short answer is that I had to download the PHPUnit PHAR, stick it into the given PHP project's top-level directory, and then point PHPStorm at it. Which I had to do every time I went to a new exercise. In JetBrains' defence, they probably weren't designing their product for people who keep switching to a new project every hour or so. Finally, though, I could actually run the supplied unit tests.

On the point of PHP Docker images, JetBrains themselves publish a Dockerfile for PHP development. At first, I was hesitant to use it, since it hadn't been updated in six months. I eventually realised, however, that it would base itself on the latest official PHP 8.2 at the time the container is built, so in fact it should be completely up-to-date for most purposes. It just adds on a few extra bits atop the official image. It actually worked pretty well once I had things figured out. Though, I can't say that PHPStorm felt as polished as say, Rider. Perhaps it was simply that I didn't get deep enough into a big PHP project to experience it, however.

My Impressions

As mentioned earlier, my impression of the language is mixed. It exceeded my very low initial expectations, but I wasn't blown away by it either.

The language just feels a bit messy to me. I think I saw a quote somewhere from the original PHP creator that he never had any sort of plan or roadmap for it, he just kept chucking in the next feature that he thought would make sense. To be honest, that does show through a bit. For one thing, there are an enormous number of random top-level functions in the global scope (rather than behind certain namespaces). I have the impression, though, that someone has been slowly rationalising the language and tidying it up over the past few minor releases or so, based on notes I saw scattered throughout the PHP documentation. I personally think this actually bodes well for the future of the language.

Another thing I didn't love was the limited range of built-in types. Probably the utterly gigantic BCL in .NET has spoiled me, but PHP has a handful of primitive types, associative arrays, and... that's about it (from what I saw, at least, though I know that Laravel has some nice extras at least). The design of the associative array is fairly reasonable on the whole, however. I'm less-than-thrilled about the fact that you can only use integers and strings as keys, but the design with integers is quite intelligent. It means that you can use an array basically like a C-style array without ever noticing that it's actually associative if you prefer. I also didn't really love that there isn't a separate char type to strings (you can reduce a string to individual characters, but those characters still have string type I believe).

On the positive side, there are some really nice utility functions in the global namespace. For instance, working dates and times is quite easy with the in-built facilities. The gigasecond Exercsim exercise reduces to two simple lines with PHP's functionality. Many of the others weren't too far off that, either. PHP also seemingly includes reasonably solid UTF-8 support on a like-for-like basis for many string handling functions—mostly, you just need to prepend the string function's name with mb_ (for multibyte).

I was also pleasantly surprised at how much support there was for typical functional concepts like higher-order functions and the usual map/filter/reduce style suspects. Using them can be a bit clunky, but they're there, and they do what you want. This was definitely one thing I wasn't expecting.

Conclusion

I don't see me rushing out to try to use PHP for just about anything, nor immediately recommending it to others. At the same time, I also don't see me trying to avoid it like the plague. Almost certainly I wouldn't choose PHP for any new project, and nor would I actively seek out jobs where working with PHP is a big part of it. Yet, equally, if I was told that there would be some PHP work in a job, that wouldn't put me off.

I'm sure there's more I could (perhaps should) say, but I can't think of it right now. I'll come back and update this post in the future if I think of anything else.

Incidentally, if you happen to be looking into setting up a MISP instance for your organisation, I strongly recommend looking into CloudMISP. I was never involved in the work on CloudMISP, but I had enough experience with it while developing a plugin for a client's product to act as a bridge between MISP and that product to know it can be a pain in the neck. The amount of hassle using the off-the-shelf SaaS product can save you is, well, it wouldn't take long before the price is outweighed by the hair not torn out of your head. ↩
At least the dynamic typing does a lot to alleviate the need for generics, making the basic type system much less underwhelming than Golang's until v1.19... ↩

Deploying Then Securing the OWASP Juice Shop, Part One-Point-Five of ?

James Cooper — Mon, 24 Jul 2023 00:00:00 +0000

Difficulties getting started with AWS

Summary

A short overview of some of the many issues I encountered when trying to get myself up and running with AWS.

AWS Doesn't Like Itself?

After creating an AWS root user account, I followed their introductory instructions to create a new AWS organisation and an Administrator user inside said organisation, in accordance with best practices (i.e. don't use your root user account for anything but administration of the overall organisation).

Unfortunately, when I tried to log in using said organisation account, all I ever got was a rejection from AWS telling me that my credentials weren't valid. So, I logged back into the root user account, and used their password reset flow to send a new password reset email to the org user. I then used the link in said email to reset the password, and tried logging in again, only to be met with the exact same result.

The only thing I can think is that the instructions AWS provided are, in some way, flawed. Or that I didn't follow them correctly, which would suggest that the instructions aren't as robust against user errors as they should be. Plus, I'm pretty sure I followed them exactly as written. As of the time of writing this, I still haven't resolved the issue, but I'm beginning to think that I will need to blow that org away and start all over again, which is a pain in the rear end.

Redoing the tutorial and duplicating everything seemed to work. Fingers crossed that it stays working...

The Next Stage Isn't Better

The next step discusses setting up the AWS CLI. After telling you to install the CLI locally, it goes straight into configuration of credentials. Which makes sense, except it dives straight into telling you what information you need, but doesn't explain anything about how to get your hands on it:

To configure the credentials, use the command aws configure and include the credentials of the user created in the previous module of this tutorial. Add the user we included in the user group with administrator-level permissions.

When you use the aws configure command, you will be asked for:

AWS Access Key ID

AWS Secret Access Key

Default Region: Provide the Region in the following format us-east-1. For a list of Region names and codes, see this table.

Default Output Format: This is how the output should be displayed by default, and includes, but is not limited to: json, yaml, text. Review the documentation for all options.

Precisely nothing about where to find the relevant credentials. Moreover, when I eventually discovered after searching through the AWS documentation that you can apparently find CLI credentials on the first page you see when you log in, I couldn't figure out how to get back to that page. Eventually, I logged out and back in (which seemed like it almost wasn't going to work, yet again...). That showed me a screen which said I could look up credentials for command line access, but it appeared that the only credentials I could access there were the ones for the root user account. Given that this is new-user-onboarding documentation that AWS themselves point you to, it's rather rubbish, quite frankly.¹ I still haven't figured out how to get CLI creds for the org user yet.

It is possible that I am completely misunderstanding something about how AWS works here, but if so it is completely unclear to me at this point. Which suggests that their documentation for beginners to AWS is seriously lacking.

AWS SSO

After eventually giving up and just using what appeared to be the creds for the admin account's CLI access, I then tried to follow the other instructions they linked to about using aws configure sso. That didn't go any better, quite frankly. While it does appear that the relevant config files have been set in my .aws folder, they just don't seem to work very well in many circumstances. E.g. I tried to use aws sso login, and just got pointed to a login webpage that kept on looping through the login process, without ever updating the CLI tool. Seriously, I went through entering my login credentials something like six times in a row, where each time after I finished it would just take me back to the start, while the CLI tool waited for a response from the API. I just gave up in the end.

Fortunately, it seemed like I could copy-paste stuff out of the web portal with the admin creds directly into the configuration file, and that seemed to work—though I also had to copy in a 'session token' field, of which no mention is made in the walkthrough. It was an absolute nightmare just to get myself set up on the basics, however, especially when I FOLLOWED THE DARN AWS BEGINNER GUIDE AND IT DIDN'T FREAKING WORK.

All in all, a very poor first impression for getting myself set up independently on AWS. Honestly, if it weren't the 800-pound gorilla² of cloud services, that would have been enough to make me run in the other direction and not look back.

Dishonourable Mentions

I came across other stuff in under the getting started sections that was seemingly, at best, badly outdated. For example:

Deploy a LAMP Web App on Amazon Lightsail,
which seemingly assumed that you were using Bash on a Linux machine. As someone on Windows using Powershell and not greatly familiar with LAMP applications, this so-called guide was rather unhelpful. Fortunately, I could use WSL to cover this, which I guess perhaps AWS assumes people on Windows are already operating in. Plus, the Lightsail blueprint and bundle IDs specified in the guide were so out-of-date as to be unavailable anymore. Fortunately, in this instance there seemed to be pretty obvious modern versions of these. Lastly, the big blob shell script that they tell you to copy and paste doesn't seem to work anymore. As at the moment of writing, I'm unsure how I can observe what's happening on the Lightsail server to find out what went wrong. I presume that it's some sort of drift between versions of the LAMP Lightsail blueprint or the sample application that the tutorial tells you to clone. I don't know nearly enough about LAMP-stack applications to make a guess at what the issue might be.³,⁴
On the Cloud Essentials page, one of the sample topics under 'Launch your first app' says "Getting Started with .NET Development on AWS with Visual Studio 2019". VS 2022 has already been out for well over a year at this point, and the latest (greatest?) versions of .NET are only supported in it. True, you probably can use 2019 for their purposes fine, but this really looks like something they need to revisit. I can understand some documentation getting a bit stale, but not stuff linked to on their introductory 'Cloud Essentials' page.
A newer Lightsail tutorial failed on the first section with problems to do with building a Docker image. In their defence, this problem seems to be something arising from changes in Debian around installing Python packages. They apparently are trying to introduce PEP 668. It's a worthy goal on Debian's part, but unfortunately it does introduce a whole bunch of pain in the neck when I just want to follow a tutorial. Trying a user installation by appending the --user flag didn't help. I tried the suggested --break-system-packages flag to pass to pip on the failing RUN command in the Dockerfile, but even that didn't work fully—it failed on a subsequent step. In the end, using --break-system-packages and specifying --python python3 to pipenv seemed to do the trick. Fortunately, it was largely much smoother sailing after that. At least, until stage 4, when they completely missed out the part where you need to use the updated container image that was just pushed to Lightsail to do a second deployment. It was trivial to change the previous instruction appropriately, but they didn't even say to do it.⁵

But You Claim You Did That Training?

I should clarify, in case anyone starts wondering: The training for the first AWS certification, AWS Certified Cloud Practitioner, that I undertook via A Cloud Guru covered all the foundational high-level concepts that you will be tested on in the certification exam. It didn't however, walk me through getting started with an AWS account. Rather, the exercises included used dummy accounts set up under their own AWS organisation, with login creds provided by them. Which worked very well for the purposes of the training, to be honest. Besides, given that AWS apparently can't keep its introductory material up to date, what hope does an outside organisation have?

Conclusion

I found that many of the AWS introductory training materials and documentation were, at best, badly outdated. In some instances, they rather seemed to assume that someone was already an AWS expert. It was not a very good impression at all. One positive thing I will say about the tutorials I saw, however, was that they pretty much all ended with cleaning up whatever resources that were stood up during the tutorial. I imagine that they probably do it so they don't need to deal with a legion of practice instances doing nothing for nobody, but it does help prevent people suddenly getting billed for something they have forgotten about down the line.

And I had always heard that AWS was supposed to be the cloud provider with excellent customer service, to boot. ↩
To misuse Dr Joe Armstrong's famous quote, it definitely feels like I'm forced to deal with the whole forest if I want to use this gorilla's banana... ↩
It appears that quite possibly that tutorial has been superseded by this one (which also seems to be a much better tutorial), but the older tutorial was the one that I was pointed to out of the gate when I had just signed up to AWS. ↩
I eventually stumbled across yet another tutorial, which seems to be a point-and-click equivalent to this first tutorial. Honestly, it seems like it's probably a much better resource given that the whole thing is just about deploying a pre-formed LAMP-stack instance to Lightsail. ↩
Actually, they did eventually get to that, but not after telling you to note that the container was being deployed, meaning I had to infer I was already supposed to have done that. It looks like a copy-paste error. Does nobody proofread these things? ↩

Deploying Then Securing the OWASP Juice Shop, Part One of ?

James Cooper — Tue, 11 Jul 2023 00:00:00 +0000

Deploying, and then Securing, the OWASP Juice Shop Application

Summary

I shall deploy the deliberately-vulnerable OWASP Juice Shop application to 'the cloud', and then use various techniques and tools to (attempt to) secure it.

Introduction

OWASP Juice Shop is one of OWASP's flagship projects, and is a deliberately-vulnerable web application. It is used to demonstrate various vulnerabilities that can exist in real applications (including the whole of the OWASP Top 10), for the benefit of all three of builders, breakers and defenders. The Juice Shop page on the OWASP website describes it this way:

OWASP Juice Shop is probably the most modern and sophisticated insecure web application! It can be used in security trainings, awareness demos, CTFs and as a guinea pig for security tools! Juice Shop encompasses vulnerabilities from the entire OWASP Top Ten along with many other security flaws found in real-world applications!

Juice Shop is written in Node.js, Express and Angular. It was the first application written entirely in JavaScript listed in the OWASP VWA Directory.

The application contains a vast number of hacking challenges of varying difficulty where the user is supposed to exploit the underlying vulnerabilities. The hacking progress is tracked on a score board. Finding this score board is actually one of the (easy) challenges!

Apart from the hacker and awareness training use case, pentesting proxies or security scanners can use Juice Shop as a “guinea pig”-application to check how well their tools cope with JavaScript-heavy application frontends and REST APIs.

While sitting in one of many excellent talks at the recent OWASP NZ Day 2023, I had the idea of using something like Juice Shop as a ready-made web application for my own purposes. Namely, to gain more hands-on experience actually deploying a web application into the cloud, and securing an existing vulnerable application. To be totally honest, the part about securing it is the thing which appeals to me more, but I figure I really should get more experience with deployment, also.

Why Juice Shop?

Why did I pick the OWASP Juice Shop application to experiment with, you ask? Basically, it met all the criteria I can think of. It:

Is written in a popular programming language that's likely to be well-supported by various tools, Node.js.
Is already a completed application, meaning that I can get straight onto deploying it, rather than spending time finishing the functionality first.
Is well-known and moreover has had considerable time devoted to exploring its various vulnerabilities, meaning that there are a lot of other references out there for me to use to learn about security issues that I don't spot myself.

Uhhh…

"Isn't deploying a deliberately vulnerable web application into the cloud a terrible idea?" you ask.

Yes, if I were planning to leave it up there for long, it would be. The plan with the deployments, however, is to leave the application up just long enough for me to confirm that the deployment was, in fact, successful. I fully intend to tear them down again shortly after, and moreover not to persist any state changes between deployments. I also have no intention of supplementing the provided data with anything else. Thus, if someone does navigate to it in the short time it is up, they will not gain access to any data they could not already access. They also won't (or, at least, shouldn't...) be able to exploit other vulnerabilities in the web app to hijack the processing power behind it for nefarious purposes.

Deployment

My initial plan is to try to deploy the Juice Shop straight onto AWS using a very clicky-pointy methodology, probably just as something running in a VM on an EC2 instance. Once I have that online and can navigate to a running instance of it in my browser, I then plan to explore using various approaches to automating the deployment further. This can include, for example, using something other than EC2 to deploy it (e.g. I believe AWS Fargate lets you deploy straight from a container without further fuss), breaking the components out into separate parts using cloud-native elements like AWS's RDS, and using Infrastructure-as-Code tools such as Terraform or Pulumi.

I also hope to explore using further cloud ecosystems than just AWS—especially Azure, but I will investigate others I come across which offer a suitable free tier.

Why start with AWS?

You might reasonably wonder why I would start with AWS of all the cloud providers. After all, they're a part of Amazon, a company which does its best to send us back to the worst days of Victorian workhouses. There are two major reasons to start with AWS: Firstly, it seems to be the cloud provider that I see mentioned the most as being a desired skill on job adverts. Secondly, it is the one I am most familiar with, having done basically everything but study for the introductory AWS certification while I was employed at Cosive.¹ I never ended up actually sitting the exam for the cert, because I was too busy working on Cosive's software, but it's still the one I feel most confident taking on first.

Security

The real main focus of this self-inflicted project is to explore securing a web application. For this purpose, I roughly plan to follow the typical flow of going from planning through to development, building and deployment etc. So, that should mean starting with exercises such as attempting threat modelling, to applying SAST tools where they offer a free version for open source, to applying DAST, to securing the cloud deployment. One other thing I specifically hope to look into is both generating Software Bills of Material (SBOMs)² automatically, the tooling around that (e.g. OWASP's DependencyCheck and DependencyTrack), and going from there to using the SLSA framework.

I'm very open to suggestions of specific other tools to explore!

Approximate Posts Plan

This post. Kicking off the whole thing by stating my goals.
Difficulties I encountered getting started with AWS.
Deploying into AWS, the most basic way(s) possible.
Deploying into AWS using alternative method(s).
Deploying into AWS using IaC such as Terraform, Pulumi or AWS CDK.
Threat modelling for the Juice Shop.
Penetration Testing: Amateur Hour
Experimenting with SAST.
Experimenting with DAST.
Cloud security (e.g. WAFs).
SBOMs & SLSA.

I fully expect this plan to be revised as I go. Moreover, as blog posts are published, I shall (try to remember to) update the plan with links to the new posts.

Future Directions

There are a number of similar projects out there that also could be used in a similar-ish fashion. Potentially including, but not limited to:³

Thanks Cosive! ↩
Apparently there are also now things like a SaaSBOM, too. ↩
See also this and this. ↩