DEV Community: 🍌🍌🍌

Hackernews clone...but for cloud news!

🍌🍌🍌 — Tue, 29 Sep 2020 18:44:36 +0000

Hello everyone on DEV!

I'm really excited to share a project I completed with you all called

Cloud Weekly ... News!

It's a news aggregator for all things related to "The Cloud"

It features news from the big three cloud vendors

AWS
Azure
GCP

As well as posts from the DEV.to community! Any posts that are cloud and devops specific get shared to the larger community and the traffic is brought back to you, the post writers.

I hope some of you will take a look and give me some feedback!

I look forward to seeing what more of you write and share with the world!

Automating Deploys with Bash scripting and Google Cloud SDK

🍌🍌🍌 — Sat, 15 Aug 2020 00:10:44 +0000

You don’t have to know terraform, ansible, chef, puppet, or any other Infrastructure-as-Code (IAC) tools to automate your deployments on Google Cloud Platform.

Google offers a several command line tools in their Software Development Kit (SDK) that can be used to create/destroy resources, interact with storage buckets, create firewall rules and more; you can automate your entire deployment!

Having a solid understanding of the SDK tools will pay dividends in your career and they’re a requirement for every GCP certificate; Its very beneficial to spend some time getting comfortable with them.

This isn’t a tutorial on how to use gcloud so if you’re generally unfamiliar with the tool, please see the resources below to fill in any knowledge gaps.

gcloud command-line tool overview | Cloud SDK Documentation
gcloud Cheat Sheet | Cloud SDK Documentation

To automate calling multiple gcloud commands to build our infrastructure I’m going to use BASH scripting. If you’re unfamiliar with BASH scripting, here’s a TLDR;

Shell commands listed in a file that get executed in sequence

Let’s look at a small deploy script.

create_instance.sh

#!/bin/bash

ZONE="us-east1-a"

echo "Welcome to our tiny script - lets create an instance"
gcloud compute instances create instance-1 --zone $ZONE

echo "Done!

This script gives us some console output to let us know its running, creates an instance named instance-1 in the zone defined by the variable ZONE in this case, us-east1-a.

With this concept we can expand it to larger more complicated deployments; and the same can be used to teardown infrastructure in an expedient programatic fashion.

To ensure our systems are properly configured once created in the cloud, we’re going to use another feature offered to us by GCP, a “startup script”. A startup script is a script that is executed once the system is created and spins up for the first time. We can configure a startup script to run a few different ways:

A remote file stored on a public server
A file stored in a google cloud bucket
Inlined into our deploy script

An example below shows us setting metadata key value pair in the gcloud command to pull a file from a remote server

gcloud compute instances create instance-1 —metadata startup-script-url=https://server.com/script.sh

script.sh

# Update and install packages
sudo apt-get update
sudo apt-get install git build-essential apache2 -y

# Update index.html page
echo "<html><body><h1>We're live!</h1></body></html>" > /var/www/html/index.html

echo "DONE!"

Our script is located on a remote (public) server, pulled down, executed, and our custom server is up and running!

This overview should give you some new insight into the power of automating network creation and system bootstrapping.

Taking the idea to the next level, I wanted to have deployment scripts for various use cases ready at my disposal when needed. One of those that I want to walk through with you today is a simple forensics laboratory. A place where files could be dissected, malware stored, and segmented from my personal and work equipment.

Here are my requirements

Forensics workstation
Kali Linux workstation
Honey pot
- Segmented from other workstations
Bucket for malware and potential bad stuff

A small artistic endeavor later this is the network diagram I’ve come up with.

This network diagram gives us more insight into how we want our resources are structured and grouped, let’s build out our design documentation more.

A VPC that contains two subnets, safe / unsafe. Safe will contain our instances used for analysis and tooling, tagged with “admin” for firewall rules that allow ingress on 22, 80, 443 and icmp (ping).

Unsafe subnet will contain one host, a honeypot, and tagged “insecure” for firewall rules to allow connections on all ports and protocols only for hosts with the appropriate tag.

Not directly an object that will live in our VPC but we’ll also have a Cloud Bucket for storage of any analysis files or reports; this resource will be a project level resource.

With these design considerations in mind and using the console UI to help us create gcloud commands we end up with the following deployment script (note: for blog post clarity startup scripts are inlined)

#!/bin/bash

echo "Create VPC and subnets"
gcloud compute networks create lab-net  --subnet-mode=custom --bgp-routing-mode=regional

gcloud compute networks subnets create safe  --range=192.168.0.0/24 --network=lab-net --region=us-east1

gcloud compute networks subnets create unsafe --range=192.168.128.0/29 --network=lab-net --region=us-east1

echo "Creating SIFT instance"
gcloud compute instances create sift-1 --tags=admin \
  --metadata startup-script='
  #! /bin/bash
  sudo su -
  apt-get update
  # TO DO: Install SIFT
  EOF'

echo "Creating Kali instance"
gcloud compute instances create kali-1 --tags=admin \
  --metadata startup-script='
  #! /bin/bash
  sudo su -
  apt-get update
  # TO DO: Install KALI Tools
  EOF'

echo "Creating Honeypot instance"
gcloud compute instances create honeypot-1 --tags=insecure \
  --metadata startup-script='
  #! /bin/bash
  sudo su -
  apt-get update
  # TO DO: Install Honeypots
  EOF'

echo "Create Bucket for storage"
gsutil mb gs://bucket-of-bad-stuff

echo "Adding firewall rules"

gcloud compute firewall-rules create allow-ingress-admin-lab-net --direction=INGRESS --priority=1000 --network=lab-net --action=ALLOW --rules=tcp:22,tcp:80,tcp:443,icmp --source-ranges=0.0.0.0/0 --target-tags=admin

gcloud compute firewall-rules create allow-ingress-insecure-lab-net --direction=INGRESS --priority=1000 --network=lab-net --action=ALLOW --rules=all --source-ranges=0.0.0.0/0 --target-tags=insecure

echo "Done"

I will leave script comprehension as an exercise to the reader!

For the lab to be complete there are still some missing pieces, none of the systems haven’t been configured and are still in their default state; which is going to be an effort for another day.

Additionally, if our instances don’t have the need to persist for extended periods of time we can cut costs by using preemptible instances, something I talk about in this blog post.

Once we’ve collected all our samples, performed our exhaustive forensics analysis, written a detailed report, and shipped it off to our client, we can start to tear down our infrastructure. This is something we can also automate with a small BASH script.

#!/bin/bash

echo "Deleting instances..."
gcloud compute instances delete sift-1 -q
gcloud compute instances delete kali-1 -q
gcloud compute instances delete honeypot-1 -q

echo "Removing bucket..."
gsutil rm -r gs://bucket-of-bad-stuff

echo "Removing firewall rules..."
gcloud compute firewall-rules delete allow-ingress-admin-lab-net -q 
gcloud compute firewall-rules delete allow-ingress-insecure-lab-net -q 

echo "Removing subnets and network..."
gcloud compute networks subnets delete unsafe -q
gcloud compute networks subnets delete safe -q
gcloud compute networks delete lab-net -q

echo "Done"

Each resource can be referenced by name and deleted using the appropriate gcloud command module. The -q flag surpasses the “Are you sure?” Warning and runs the action.

As a last action I was curious on how long it takes to setup and destroy this VPC I ran some not so scientific timing tests.

On average VPC and resource creation was completed in under 1m30s*

bash DEPLOY_FORENSICS_ENV.sh  7.95s user 1.65s system 11% cpu 1:22.75 total

*Note: Creation times do not take into account system setup time, if there are any startup scripts configured, your system may be instantiated but not fully ready for action.

Destruction took longer.

bash DESTROY_FORENSICS_ENV.sh  7.81s user 1.69s system 7% cpu 2:10.41 total

These times will also vary based on things like: machine type, if there are any shutdown scripts configured and disk size.

I hope this gives you an idea of what’s possible with automated deployments and that you’ll begin thinking about building your own disposable infrastructure to accomplish tasks rather than being tied down to owning a small fortunes worth of hardware.

Using this one simple trick you can cut your GCP compute costs by as much as 80%!

🍌🍌🍌 — Wed, 12 Aug 2020 05:18:38 +0000

Okay! Now that I got your attention let me tell you how you can lower your compute costs with minimal change to your existing infrastructure or deployment scripts.

Use Preemptible Instances!

Preemptible instances are type of compute engine resources that run at a much lower price than normal instances; they are ephemeral and are terminated (preempted) after 24 hours, if not sooner.

From the google documentation:

Preemptible instances are excess Compute Engine capacity, so their availability varies with usage.

Preemptible instances offer us massive savings as compared to normal instances of the same machine and hardware specifications. Take a look at the N1 standard machine types pricing page and compare the Price and Preemptible price columns.

EIGHTY SEVEN PERCENT !!!

Now you’re probably thinking “Where do I sign up? How do I move everything over to preemptible instances?”

Slow down! They’re not great for everything and shouldn’t be used in systems where downtime is unacceptable. Here are some situations where using preemptible instances have an advantage and could cut down on operating costs.

Scientific, medical, or engineering efforts that can tolerate loss and addition of computation nodes without disruption to the overall process.
Batch image or video rendering workloads where processing power is needed on demand and where jobs that don’t exceed 24 hours.
Machine Learning modeling that can be distributed across nodes
Other applications that can restart from a saved state (file).

On Google Cloud Platform there are three ways to utilize preemptible instances on your compute resources:

Single compute instance with preemptible option set.
Managed Instance Groups - The configured compute engine template must have the preemptible option set.
GKE Kubernetes clusters can also leverage preemptible instances as the underlying compute stack.

This sounds like all positives, so let’s talk about the downside, being preempted, aka, having your machine terminated. When its been determined that your instance is headed for destruction you are given a 30 second window to execute any clean up functions, data synchronization, and get out of there before the machine is forcefully deleted. You can accomplish this by configuring a shutdown script, which will automatically be executed when the system receives the signal. The script as part of its execution triggers a shutdown of any services you want and copies files to a remote location, like a cloud bucket for the next restart. Below is a link to a great sample of a shutdown script and a python script catching the event for a graceful exit.

Running shutdown scripts on Compute Engine
Sample Shutdown script for preemptible instances

Preemptible instances aren’t a panacea for your billing woes; they aren’t ideal for all workloads; availability may become a concern. This is just one of many ways to cut your cloud costs and one of the first steps to knowing which direction to go in is to understand what you’re trying to do (knowing what direction to go in helps too).

Enjoyed the post? Let me know! 💛🦄🔖

I’m a certified Associate Cloud Engineer!

🍌🍌🍌 — Fri, 07 Aug 2020 19:56:06 +0000

It’s official everyone!

I passed the Google Cloud Platform Associate Cloud Engineer exam!

I’m proud to have gotten this certification and I wanted to share with you what the test is like, my thoughts on my experience, and what my methods and tools were that helped me achieve success and pass on my first attempt.

About the exam.

Exam format: - Multiple choice and multiple select, taken in person at a test center.
Length: 2 hours
Registration fee: $125 + taxes ≈ $137, There is also a discount for those from countries with lower purchasing power parity.
Recommended experience: - 6 months+ hands-on experience with GCP.

My experience and thoughts

The test and studying was a fantastic primer into the world of managed services and cloud technology. The core concepts needed to succeed in “the cloud” are (more or less) the same as traditional/enterprise security.
Technical concepts like, subnetting, CIDR ranges, least privilege principle, and others are things I learned throughout my career and without a decent understanding the learning curve is going to be steep.

I can honestly say even with all the preparation I did the day of the exam I still felt nervous; I guess that means it was important to me. You can take the exam in person, or digitally. I opted for a in person exam because the next virtual exam was weeks away and I didn’t want to wait that long.

Due to the pandemic we’re still facing in the United States I didn’t expect there to be so many people taking exams! I was walked into a room with cubicles and workstations, there were many other people in the room taking various other exams, and I was pleased with the health and safety precautions that were taken by the testing center.

I finished the exam in ~45 minutes and when I was done I felt confident in my answers, but not necessarily that they were correct. I expected a grade once I clicked “submit exam” but the next window was a feedback form, which honestly, how can I give you feedback, I’m anxious about the score!?! I was presented with the 4 characters that lifted so much weight off my shoulders, PASS. There was no indication of my score, what I got wrong, what I got right, just a pass or fail grade; the result also isn’t “official” for 7-10 days while Google performs their validation. Minutes later an exam proctor walked me out and sent me on my way.

Two days later I received an email saying my exam score was accepted and I received a digital certificate. I was also added to the Google Cloud certificate holder website which was a nice added surprise!

Jason Schorr - Google Cloud Credential Holders

What I used

LinuxAcademy was a huge contributor to my exam success. The videos total 14 hours with 7 hands on labs to get practical experience. At the end of the course there is a Practice Exam which I made sure to get to the point where I was getting 100% consistently, its not enough to know the right answer, but to also understand why. I didn’t use the labs provided with LinuxAcademy, but hands on practice is critical to success; I had lab access through another service, Qwiklabs.

Course: Google Cloud Certified Professional Cloud Security Engineer | Linux Academy

Google offers practice exams for their certs and I highly recommend going through it until you’re getting 100% - the wrong answers are highlighted and the correct answer is supplied along with the reasoning behind them.

Practice exam offered by Google

Earlier this year Google was offering free Qwiklabs training for anyone who signed up via their links. It cost me nothing, was extremely quick to get through, and allotted me 3 months of unlimited training. This hands on training allowed for operational experience that was instrumental to exam success. There are questions of the exam where your operational command line knowledge is tested.

Qwiklabs provides real cloud environments that help developers and IT professionals learn cloud platforms

Although the Google offer for 3 months of lab time has ended, keep an eye on the medium article below, Sathish VJ updates it often with some codes for free months; also his training material is also top notch!

Watch this for updates to get for qwiklabs credits

Graces flowcharts is a fantastic resource for really understanding the situations you would want to use a specific service for. Do you understand the difference between BigTable, BigQuery, Spanner? Do you know when you should use a SSL Proxy vs HTTPS load balancer? I didn’t and these flowcharts helped me make sense of what to use and when.

Grace’s flowcharts

Finally we have a collection of study notes. I don’t know who compiled them but they’re verbose and cover everything you’ll need to know for the exam. I didn’t find these notes until a few days leading up to my exam so they were really good as a quick reference and having links to the official G docs. I wouldn’t use this doc on its own but its a great thing to have as an adjunct to your studies.

Associate Cloud Engineer - Study Notes - Google Docs

Now what?

Now I start preparing for the next certification I want to get and that’s the GCP: Professional Cloud Security Engineer. Based on my skillset and experience I think its the most logical step with the least amount of new material to learn. Diving deeper into the nuts and bolts of GCP is exciting and I’m looking forward to gaining a more complete understanding of the tools and options available to us.

Hey reader!

Do you have any questions on my experience?
Did I cover everything?
What do YOU want to know?

Enjoyed the post? Let me know! 💛🦄🔖

Social Media Accounts Shouldn't Be Static! 💃🏽

🍌🍌🍌 — Wed, 22 Jul 2020 02:52:42 +0000

Inspired by other posts across the Internet

This Video Has 15,608,459 Views - YouTube

This Post Has 2,233 Views, 154 Reactions And 23 Comments - DEV

I decided to try to accomplish the same on Twitter with my username field and the number of followers I have.

The process is pretty straight forward, lets outline it

1. Connect to twitter API
2. Get my current number of followers
3. Update my username field
4. Repeat every 10 minutes

Now that we’ve outlined the process lets breakdown the code and infrastructure

For this project I decided to use Python3. I really like using it for quick scripts over node.js because I don’t want to deal with asynchronous code; its nice to have code that runs one line, after the next, with no interruptions.

You will need Twitter API credentials for this project, if you don’t have them yet, you’ll have to apply for them at https://developers.twitter.com.

To connect to the Twitter API I’ll be using Tweepy. It’s great, it works easily and has built in support for Twitter API’s rate limiting.

For hosting and scheduling I like to use Heroku. Heroku allows you to run your code and have an exposed web service, capable of handling web requests and serving up content. In this project we won’t be handling any external request, nor serving up data, but we will still need to make sure our code plays well with Heroku, so we’ll be sure to add a web server, in this case Flask easily fills the gap.

For our code to run on a schedule or every X minutes, we will enable a Heroku add-on for our project, Heroku Scheduler which once configured will launch our script on our defined schedule.

Awesome, let’s dive into the code!

import tweepy
from flask import Flask

First up we have our import statements, these statements allow us to use functionality provided by 3rd party libraries.

The os library gives us access to the underlying operating system and allows us to run commands directly on the host.

Tweepy is going to enable us to connect and communicate with the Twitter API.

Flask is a micro web framework enabling us to handle http requests and many other features, for this project we’re using it to satisfy heroic deployment needs.

Next up is authentication

# Authenticate to Twitter
auth = tweepy.OAuthHandler("consumer_key", "consumer_secret")
auth.set_access_token("token_key", "token_secret")

We need two sets of keys to authenticate, both can be found on the twitter developers page on your app, similar to the screenshot below.

Once you’ve added those keys, the next step is connecting to the API server and getting some data.

# Create API object
api = tweepy.API(auth)

#get my user object
iam = api.me()
followers_count = iam.followers_count

The me() method of the tweepy object connects to the twitter API and pulls down the current authenticated users user data.

The user data object thats returned to us can accessed by using the dot . operator, in this case we want the followers_count attribute.

If you don’t know the attributes of an object you can use these builtins:

dir(object)
vars(object)
__dict(object)__

Combine that with PrettyPrint for some legibility:

BONUS CODE SNIPPET

import pprint
pp = pprint.PrettyPrinter(indent=4)
pp.pprint(vars(object))

Now that we’re done with that little detour, let’s get back on track.

The next part of the code should update our username

 api.update_profile(name=“Jason Alvarez is {} new friends away from 10k".format(10000 - int(followers_count)))

Tweepy provides us a function called update_profile which will update our profile depending on the parameters supplied to it. In this case we’re passing the name parameter and giving it the following string

“Jason Alvarez is {} new friends away from 10k”.format(10000 - int(followers_count)))

Rather than insert just a number, I decided to use the space as a countdown to a specific target.

Format takes a string with placeholders {} and inserts the values supplied (value1, value2…)

string.format(value1, value2…)

This same technique can be applied to any editable field in a profile and any data that the API won’t reject for input; please refer to Docs — Twitter Developers for more details on valid and invalid characters.

The last segment of code is needed for the script to gracefully run on Heroku’s infrastructure. While not specific for this project, let’s go over what it does and what it’s not doing.

app = Flask(__name__)
app.run()

Flask as mentioned above is a micro web framework, it’s used normally to create servers that handle http(s) requests and other really neat things. In this case we need to start a server so that Heroku knows our deployment was successful.

One of the big uses for Flask, is to create your own API Microservices, where you can define your own HTTP routes, a feature we are not using in this case, and is outside the scope of this writeup but I highly suggest you look into it!

OKAY THE CODE IS DONE!

Great! We’ve finished writing our script and that means we’re all done and can deploy, right? ALMOST!

To deploy to Heroku successfully we need two more files:
requirements.txt
Procfile

requirements.txt outlines the libraries needed for our code to run successfully and this isn’t something you need to create manually.

To generate your requirements.txt run this command in your project directory pip3 freeze > requirements.txt

From Heroku’s documentation

Heroku apps include a Procfile that specifies the commands that are executed by the app on startup.

Our’s is going to be real simple and just launch that web server we declared in our script.

Edit your Procfile so it has the following contents
web: FLASK_APP=update_username.py flask run --port $PORT --host 0.0.0.0

$PORT will be supplied to us by the operating system shell environment, and will be random, so don’t try and set anything here.

Great, if you’ve made it this far your directory should look like the following

.
├── Procfile
├── requirements.txt
└── update_username.py

NOW WE CAN FINALLY DEPLOY!

Create a new project on heroku.com and follow the onscreen instructions, it’s a lot like setting up a GitHub repo.
* Create a project
* init
* add
* push
(Follow their instructions for more details)

If your push and deploy were successful, you should see something like the following in your terminal

app[scheduler.5678]: * Running on http://0.0.0.0:48547/ (Press CTRL+C to quit)
heroku[scheduler.5678]: State changed from starting to up
heroku[scheduler.5678]: Cycling
heroku[scheduler.5678]: State changed from up to complete

At this point your Twitter user name should now be updated with the string you used in the script. In my use case I want my user name to be pretty in sync with the number of followers I have, so let’s set it up to run on a schedule.

To accomplish this we’re going to use a Heroku project add-on called “Heroku Scheduler”, it’s free and shouldn’t bump up the cost of your dyno, add it to the project, then click it to configure the jobs.

To stay as in sync as I can I chose a “Every 10 minutes” job and for the run command, I used the same content that’s in the Procfile -
FLASK_APP=update_username_num_followers.py flask run —port $PORT —host 0.0.0.0

Save the job and you’re all done.

Let’s go over what we’ve accomplished.

1. We have a script that will update our Twitter username
2. It communicates with Twitter API for data
3. Our code project is deployed on a managed service

Thats not bad for a little fun script!

Enjoyed the post? Let me know! 💛🦄🔖

🚀TOP Beginner ReactJS Resources 🎊2020🎊🚀

🍌🍌🍌 — Fri, 10 Jan 2020 03:36:47 +0000

React is a JavaScript User Interface library for creating interactive web applications.

These are the resources I used to learn to write clean efficient code!

create-react-app

Create React App works on macOS, Windows, and Linux. Create React apps with no build configuration.

The Beginner's Guide to React by Kent C. Dodds

This course is for React newbies and anyone looking to build a solid foundation. It’s designed to teach you everything you need to start building web applications in React right away.

https://egghead.io/courses/the-beginner-s-guide-to-reactjs

React Bits

A free book that talks about design patterns/techniques used while developing with React.

https://vasanthk.gitbooks.io/react-bits/

🚀Absolutely Awesome React Components & Libraries 🚀

Curated List of React Components & Libraries.

https://github.com/brillout/awesome-react-components

These resources are ones I go back to regularly to and were instrumental in getting me from zero to writing fully functional and responsible web applications.

What stuff did I miss?

Enjoyed the post? Let me know! 💛🦄🔖

Bucket Hunting 101 - Bounties, Glory, and Fun!

🍌🍌🍌 — Thu, 02 Jan 2020 00:37:20 +0000

As the internet grows and services become increasingly “server-less” one of the big issues facing companies and startups are cloud security misconfigurations. Servers now can be spun up, torn down, and scaled with a few clicks, as a result security can be overlooked.

Here are some common bucket uses:

Publicly accessible data - Google LandSat
Phone or Web App storage - Instagram, Homeroom, Cluster
Websites - RottenTomatoes, IMDB

There are many tools that exist for bucket hunting the ones below are my favorite methods for finding open buckets.

Amazon S3 Buckets

S3 was launched March 14, 2006 and is currently the largest datastore on the internet. The current URL format for all S3 buckets is
https://<BucketName>.s3.amazonaws.com

It’s simple enough to write your own script to brute force the existence of a bucket with a GET request, but rather than reinvent the wheel let’s use vetted and weathered tools.

S3Scanner

A simple and effective brute forcer that permutes user supplied wordlist, checks for the existence of, and dumps the contents of the bucket.

GitHub - sa7mon/S3Scanner: Scan for open AWS S3 buckets and dump the contents

Bucket-Stream

Find interesting Amazon S3 Buckets by watching certificate transparency logs. This tool listens to various certificate transparency logs and attempts to find buckets from permutations of the certificates domain name.

GitHub - eth0izzle/bucket-stream: Find interesting Amazon S3 Buckets by watching certificate transparency logs.

Google Cloud Buckets

Google’s cloud service offerings and environment grows daily. They offer free credits for new users and incentives for small businesses. The URL structure doesn’t follow the same pattern as S3 and as such there are no certificates to listen for, more computationally expensive methods are needed.
http://storage.cloud.google.com/<BucketName>

GCPBucketBrute

A script to enumerate Google Storage buckets, determine what access you have to them, and determine if they can be privilege escalated.

GitHub - RhinoSecurityLabs/GCPBucketBrute: A script to enumerate Google Storage buckets, determine what access you have to them, and determine if they can be privilege escalated.

Microsoft Azure Buckets

Not wanting to be left out of the race, Microsoft started their Azure cloud early 2010. Offering as many if not more services as its competitors, its bucket url’s are harder to find and enumerate. Fortunately there are tools and documentation that can help us. The URL format is as follows:
https://<ACCOUNT>.blob.core.windows.net/<CONTAINER>/<BLOB>

Brute forcing multiple parts of a string together is computationally complex, so we'll break it up into smaller parts.

For brute forcing we are going to use gobuster to help us enumerate DNS and directory entries.

GitHub - OJ/gobuster: Directory/File, DNS and VHost busting tool written in Go

Starting from smallest set of enumeration targets to largest set

Account < Container < Blob

Brute force Account
Use gobuster DNS module
gobuster -m dns -u “blob.core.windows.net” -w
Brute force Container
Use gobuster DIR module
gobuster -m dir -u “.blob.core.windows.net” -e -s 200,204 -fw
Finding BLOBs
If listing is enabled you can use the same endpoint with some added query parameters.
https://<ACCOUNT>.blob.core.windows.net/<CONTAINER>?restype=container&comp=list

Otherwise you can attempt to brute force filenames using gobuster. (Not recommended)

Digital Ocean Spaces

A relative newcomer to the space digital ocean has a “S3 compatible” system they call spaces, and with this compatibility comes the same issues and enumeration methods as S3.
https://<BUCKETNAME>.<DATACENTER>.digitaloceanspaces.com

Digital Ocean offers 5 different global data centers which impact the URL

sfo2 - https://<BUCKETNAME>.sfo2.digitaloceanspaces.com
nyc3 - https://<BUCKETNAME>.nyc3.digitaloceanspaces.com
ams3 - https://<BUCKETNAME>.ams3.digitaloceanspaces.com
sgp1 - https://<BUCKETNAME>.sgp1.digitaloceanspaces.com
fra1 - https://<BUCKETNAME>.fra1.digitaloceanspaces.com

Bucket names can contain [a-z][0-9][-] and must be between 3 - 63 characters in length.

Spray & Pray

Don’t care about targeting a specific platform? These tools below will provide you a plethora of data covering (maybe?) every cloud storage provider. Don’t expect them to work the same or as well as some of the tools above but they are great when targeting specific websites.

LolrusLove - Spider for Bucket Enumeration (AWS S3 Bucket, Azure Blob Storage, DigitalOcean Spaces, etc…) - Alpha v0.0.6 · GitHub

GitHub - jordanpotti/CloudScraper: CloudScraper: Tool to enumerate targets in search of cloud resources. S3 Buckets, Azure Blobs, Digital Ocean Storage Space.

Visualization

All of the tools listed above are expected to be executed from the command line; the power users of these services interact with them via the command line; yet the overwhelming majority of the files stored need a GUI to be viewed for content and context analysis.

Bucket Miner brings you that capability, enter a bucket name and if it is publicly accessible will be presented to you for easy and intuitive viewing.

https://miner.datadrifter.xyz

These tools should get you started hunting for open buckets. Did I miss a tool you like or some new methods I didn’t talk about? Comment below and let’s chat!

Enjoyed the post? Let me know! 💛🦄🔖

Easy and FREE ways to publish a website in 2020!🥳

🍌🍌🍌 — Sun, 22 Dec 2019 03:19:14 +0000

There are too-many-to-count ways to publish a website in 2019 and there will be more in 2020. I've scoured the internet for free web hosting and deployment services. I've tried many and these are my personal picks for easy and FREE web hosting in 2020!

GitHub pages - https://pages.github.com/

Websites for you and your projects. Hosted directly from your GitHub repository . Just edit, push, and your changes are live.

Free site hosting, custom domain support and free SSL certificates to show you care about security!

surge.sh - https://surge.sh

Simple, single-command web publishing. Publish HTML, CSS, and JS for free!

Don’t like to leave the command line? Fear not! Surge.sh to the rescue. Easy publishing without leaving your terminal.

Zeit - https://zeit.co

ZEIT Host your web projects with zero configuration, automatic SSL, and global CDN.

ZEIT is a cloud platform for static sites and Serverless Functions. Easy, free, what more do you want?

Netlify - https://netlify.com

Deploy modern static websites with Netlify. Get CDN, Continuous deployment, 1-click HTTPS, and all the services you need.

I’m a fan of how simple it is to deploy a site with netlify. Connect to your GitHub repo or drop a folder onto their website; super simple!

AWS Amplify - https://aws.amazon.com/amplify/console/

Hosting for fullstack serverless web apps with continuous deployment_A fantastic and free (for a year) service from Amazon.

If you’re familiar with AWS products you’ll love amplify, integrates with the rest of the amazon stack; do it all from one browser tab!

Google Cloud Platform - https://cloud.google.com

Build, Test & Deploy With Ease. Get $300 To Try Google Cloud Now. Deploy in Minutes.

Google cloud platform is overkill for small and simpler projects. If you need server less services like, databases or lambda functions; GCP is a solid choice.

--
Enjoyed the post? Let me know! 💛🦄🔖

Building a url shortener for fun and no profit!

🍌🍌🍌 — Sat, 21 Dec 2019 22:47:01 +0000

This was a fun full stack project weeknight project and I learned a lot while doing it and would love to share the process with you all!

When starting any project it we should start by working through the SDLC to maximize our productivity and minimize wasted time.

Stage One - Planning

Q: What are we trying to build?

A: A URL Shortener.

Q: Why?

A: It's a fun project!

Stage Two - Analysis

Q: Is this something people would use?

A: Totally! There are many out there, but this one is mine.

Q: Potential for abuse?

A: Yes, must consider mitigations.

Q: Am I worried about monetization?

A: Not so much.

Stage Three - Design

Let's think about what we want our user experience (UX) to be once they land on our page. I want this to be simple and easy

Front End

Going for a very simple google-esque design and using a design mockup tool I settled on something like this.

Back End

The backend is the workhorse of this application, it has TWO main jobs.

Shorten Links
Unshorten Links

def shorten(val):
    # Take a "long" value and return a "short" value
    pass

def unshorten(val):
    # Take a "short" value and return its corresponding "long" value
    pass

All of our data will be stored in a database and for our purposes I chose to use SQLite, although for this implementation any database where you can have a unique primary key for every entry will work fine. If you want to use another method, you're on your own. For our short url purposes we only want to have valid, non special characters; upper and lowercase alphabet a-z,A-Z and numbers 0-9. This gives us a key space of 62 characters or base62; our database uses just numbers, or base10.

Cool! Heavy lifting done!

Long to short => base10 -> base62

Short to long => base62 -> base10

from baseconvert import base

def shorten(val):
    # Take a "long" value and return a "short" value
    return base(val, 10, 62, string=True)

def unshorten(val):
    # Take a "short" value and return its corresponding "long" value
    return base(val, 62, 10, string=True)

Lets add some API endpoints using Flask

from flask import Flask
import sqlite3

app = Flask(__name__)
db = sqlite3.connect('file.db')

@app.route('/add', methods=['POST']):
def add_url():
    # Add url to database & get ID
    row_id = db.add(url).lastrowid
    # shorten
    short_id = shorten(row_id)
    # return to user
    return short_id

@app.route('/<string:shortURL>', methods=['GET']):
def get_url(shortURL):
    # lengthen row id value (shortURL)
    row_id = unshorten(shortURL)
    # get url from database
    row = db.get(row_id)
    # return to user
    return row['url']

We've got our big pieces designed out and some psudocode written; it's time to move to the next stage!

Stage Four - Implementation

There is no public repo for my code but check out the running implementation here - https://bnon.xyz

Front end is hosted for free on netlify, backend & database is hosted on Digital Ocean on small droplet.

Security was kept in mind during the implementation of the project. Recaptcha is used to minimize automated shortlinking and of course this is validated on the backend as well as front.

A feature I would like to add at a later date is VirusTotal link rating to mitigate any malicious actors using this service to deliver payloads or bypass network filtering devices.

Stage Five - Testing & Integration

I spent an afternoon testing and securing this application. This was a really fun project and I'm receptive to any and all bug or vulnerability reports, there's always something to learn and improve on.

Stage Six - Maintenance

So far so good, I've knocked down all the items and bugs on my list. Did I miss any?

Enjoyed the post? Let me know! 💛🦄🔖

#30DaysOfThreads - The Cyber Attack Lifecycle

🍌🍌🍌 — Fri, 13 Dec 2019 02:14:29 +0000

The Cyber Attack Lifecycle describes the actions taken by an attacker from initial identification and recon to mission complete. This helps us understand and combat bad actors, ransomware, and others.

Let’s break down the steps !

Initial Reconnaissance 🔎

Intruder selects target, researches it, and attempts to identify vulnerabilities in the target network. Some things attackers use and look for:

Whois
Target IP Ranges
Web Properties, Domains & Subdomains
Open Cloud Buckets
Google dorking

Initial Compromise 📬

Attacker compromises a vulnerable host. This may be a DMZ host or something in a higher security group via email phish. This is the first step into a network and why security people always say:

Don't click email links!
Don't open email attachments!

Establish Foothold 🧗🏼‍♀️

A compromised system is good, one that you can access is even better. Initial access or a foothold is an attackers first steps in your network. If there are network rules to block various network traffic, the attack may die here.

Escalate Privileges 📈

Attackers often need more privileges on a system to get access to more data and permissions: for this, they need to escalate their privileges often to an Admin.

Internal Recon 👀

Where are we internally , what are we looking for, and how can I get there?
Here we apply the OODA loop - a simple strategy to help you find your way forward.

Observe - What do I see
Orient - Where am I
Decision - What do I need to do?
Action - DO

Move Laterally 👣

Once they’re in a system, attackers can move laterally to other systems and accounts in order to gain more leverage: whether that’s higher permissions, more data, or greater access to systems.

Maintain Persistence 🏠

Being able to return to networks again and again is one of an attackers main goals. They may not find what they’re looking for during in the first compromise and they will want to return.

Repeat (4-7) until (Mission) Complete 🔁 ✅

Mission complete can be any number of things, anything your mind can think up from any spy or heist movie. Real data gets stolen every day. The current “average time to detect a breach” is 197 days.

Stay safe out there!

--
Enjoyed the post? Let me know! 💛🦄🔖

Starting off #30DaysOfThreads talking about the Software Development Lifecycle (SDLC)

🍌🍌🍌 — Thu, 12 Dec 2019 06:04:23 +0000

Programming is just one part of software development and having a plan beats not having a plan.

Stage 1 - Planning

We start with figuring out what we’re trying to do.

Who will be using the app?
How will it be used?
What data will be collected?

Planning is the most important phase of the lifecycle. Here changes are the easiest & get exponentially more difficult as we go.

Stage 2 - Analysis

It’s great to have an idea and plan a project, it’s better to know if people are interested and willing to pay for it.

Poll your communities, post on forums, even setup a landing page with a email form to gather leads and measure interest.

Stage 3 - Design

Design is how you envision your app to look and what your user experience will be like.

You know what data you’ll be collecting and presenting so now’s the time to make that experience be as easy and enjoyable for the user as possible.

Stage 4 - implementation

DON’T START HERE! This is the part everyone can’t wait to jump into, actually making something.

By this point you now have a carefully considered, thoughtful, and peer reviewed idea and design. Assemble your team, plan milestones and get to work!

Stage 5 - Testing & Integration

Your app is completed! Does it work the way you expect? This stage, testing, commonly called QA (quality assurance). Bugs get sent back to developers to review and fix.

You can find more information if you look up “unit testing”.

Stage 6 - Maintenance

Now that your app has been released there isn’t much to do other than make sure it continues to work as expected. Accept bug and vulnerability reports and focus energies into other aspects surrounding your app; marketing, sales, user acquisition, etc.
That’s the end of this thread! I hope you learned something!

Comments and questions below!

--
Enjoyed the post? Let me know! 💛🦄🔖

New Site Made With HUGO!

🍌🍌🍌 — Wed, 11 Dec 2019 01:19:09 +0000

Hugo is the world's fastest static website engine. It's written in Go and is used to generate all the HTML, JavaScript, and CSS used to display this site. Using a simple template syntax you can quickly create gorgeous websites with minimal code. In fact, every content page on this site is written in Markdown and then rendered by HUGO before being minifed for production.

Feeling constantly behind the curve I spent the day reading and learning about Static Site Generators and what once was old is now new again.

We've grown accustomed to dynamic sites and their many engaging features:

Comments
Up/Down Votes
Likes
Retweets/Shares

These features make sites fun, interactive, and increase the likelyhood of return users; they also provide a wonderful attack surface for abuse from trolls and other bad actors.

Not needing many interactive elements, just a place to write out my thoughts, projects, and experiences, trying out a static site generator seemed like a great idea. I also didn't want to reinvent the wheel so I used a theme provided by the community and tweaked it to my liking.

After following the HUGO quickstart setting the rest up was a snap. I'm taking this as an opportunity to try out netlify as a hosting service. They have some interesting featuers and options; at the moment I push any site changes to git and netlify picks it up from there.

Find my blog at 👉👉👉 https://0xbanana.com

Super quick, super easy!

--
Enjoyed the post? Let me know! 💛🦄🔖