I've been pulling things apart to find out how they work since I was a kid. Deconstructing a photocopier with a crowbar was a decidedly one-way process, but revealed so many interesting gears, motors and mechanisms: everything that made the machine work, just sitting beneath the surface. Software is really not all that different.
All software can be studied, pulled apart, tinkered with and understood (no crowbar required), but JavaScript in a modern browser makes this incredibly easy. It's simply a matter of having a goal and figuring out how the relevant parts work.
Story time: Be me, apparently a rebel
Many moons ago, a small company I worked for was being absorbed into a large network of agencies. Along with the mandatory switch to a time tracking application from the dark ages, everyone at the company was required to complete a handful of web-based security learning modules that took 40 minutes a piece.
Each module was a mix of reading, unskippable video content and unskippable interactive "puzzles", eventually followed by a quiz full of questions like "when can Alice and Bob write down their passwords?", and "should Charlie take these confidential documents home?". Pretty much your typical mandated corporate education experience.
Being an enterprising software developer, I made it at most 10 minutes into the first learning module before popping open the browser's developer tools and having a poke around. Several hours later, I'd finished the remaining modules and coincidently possessed scripts to uh...save valuable developer time:
- Mark the current lesson as complete, set a random but sane session time and submit.
- Mark the current evaluation/quiz as complete and 100% correct, set a random sane session time and submit.
- Jump to the next page when unskippable content has disabled the "next" button.
My team mates were interested and thought the scripts were great. My boss overheard and also thought it was great, but maybe only for the dev team. While I didn't distribute it myself, by the end of the day the script had made its own way around several other teams through word-of-mouth alone.
Everyone saved a lot of time.
It was good.
A week or so later, the owner announced that someone had finished a test in record time! Unfortunately the new people upstairs couldn't tell most of the real results from the forged ones and lacked a sense of irony, so everyone was required to do their security training again.
I don't recall ever re-taking the tests myself, but the number of times I've been identified as "the guy who hacked the security quiz" in subsequent years suggests others did have the misfortune of re-visiting the complete learning experience.
Obvious disclaimer - don't mimic this as your employer might not find your antics as amusing!
Moral of the story
- If you can perform an action on a website, you can automate it.
- If a website knows something, you can access it.
- If a website sends something back to the server, you can make the same requests.
While this story is on the cheeky side, there are plenty of useful and harmless ways to leverage your power as the client. Tinkering like this is also a fun way to level-up your debugging and static analysis skills! Here's a few of my other adventures:
- Automatically listing all of my Steam Trading Cards at their market rate
- Exporting lists of AliExpress orders as CSV
- Exporting the entire history of Tumblr messenger conversations
- Automating repetitive billing in a slow and clunky timesheets web app to be one click
- Cloning Jira tickets with a templated name that includes the current date
- Populating fields in a Jira ticket with values from a Google Sheets document
- Archiving data from an old social network before it disappeared in 2013
Starter kit
If you're interested in trying this yourself but aren't sure where to start, here are a few pointers:
- Start by observing how the existing code works: inspect elements, find relevant looking attributes on DOM nodes, see how the DOM changes with UI interactions, see what triggers network requests, what the requests and responses look like, etc.
- Use the search tool in the Chrome dev tools to search for unique-enough strings that might appear in scripts. Element ids, classes and text labels are ways to find relevant code:
- Chrome's pretty-print button in the sources pane is fantastic for making minified code readable and debuggable:
Built-in JavaScript functions are generally all you need these days for tinkering.
querySelector,querySelectorAllandfetchare your friends.Use Sources -> Snippets in Chrome or Scratchpad in Firefox to write anything more than a one-liner. The JavaScript console is great for probing, but doesn't work well for editing bigger chunks of code:
Happy hacking!
Appendix
Below are some useful snippets I find myself using to automate other people's pages. There's nothing particularly special here, but some of it might be novel if you haven't used JavaScript in this manner before.
Waiting for the DOM
Sequencing programmatic interactions with a UI almost always calls for timeouts or condition checks to ensure the page is ready for the next action. These are two functions I use in almost every script:
/**
* Timeout as a promise
*
* @param {int} time - time in milliseconds to wait
* @return {Promise}
*/
function timeout(time) {
return new Promise(function(resolve, reject) {
setTimeout(resolve, time)
});
}
/**
* Return a promise that resolves once the passed function returns a truthy value.
*
* @param {function() : bool} conditionFunc
* @return {Promise}
*/
function wait(conditionFunc) {
return new Promise(function(resolve, reject) {
var interval;
interval = setInterval(function() {
var value = conditionFunc();
if (value) {
clearInterval(interval);
resolve(value);
}
}, 100);
});
}
Getting the DOM content before script execution
Some pages are served with useful information in their HTML that gets stripped out when the page's own scripts run. To get around this, you can fetch a copy of the original HTML from the server and use DOMParser to get a fully functional DOM context to explore without scripts interfering:
/**
* Get a DOM node for the HTML at the given url
* @returns HTMLDocument
*/
async function getDom(url) {
var response = await fetch(url, {
mode: 'cors',
credentials: 'include',
});
// Grab the response body as a string
var html = await response.text();
// Convert HTML response to a DOM object with scripts remaining unexecuted
var parser = new DOMParser();
return parser.parseFromString(html, 'text/html');
}
Scripting across page loads
When the target site requires full page loads to perform actions, an iframe can be used to avoid page changes interrupting your code. Provided the X-Frame-Options header is absent or set to sameorigin on the target pages (fairly common), the original page can be used as a platform to access other pages on the same domain:
var client = document.createElement('iframe');
client.src = window.location;
document.body.appendChild(client);
// Do stuff in the iframe once it's loaded
client.contentDocument.querySelector('a').click();
Getting data out
Copy-paste
The cheap and cheerful way to get text data out of a page is using prompt() and copy-pasting from the dialog:
prompt('Copy this', data);
File download
If you have a large amount of text or binary data collected in a variable, you can download it using file APIs:
/**
* Download the contents of a variable as a file
*/
function downloadAsFile(data, fileName, contentType='application/octet-stream') {
var file = new Blob([data], {type: contentType});
// Make the browser download the file with the given filename
var node = document.createElement('a');
node.href = URL.createObjectURL(file);
node.download = fileName;
node.click();
}
HTTP request
On pages with poor or missing Content Security Policy settings, you can simply POST data to your own server as an HTTP request. This tends to only useful if you want to export a ton of data directly into a database without double handling it.
fetch('https://myserver.example.com/ingest-handler', {
method: 'POST',
mode: 'no-cors',
body: data
});
This works regardless of cross-origin-request headers as an HTTP client has to send the whole request before it sees any response headers.
Top comments (2)
It was already an interesting read and then I saw you included some code and examples so others can get started. Some of these I didn't know myself. Great post!
After listening to too many Security Podcasts, I am kinda' interested in it myself!
What a great read!