APIs with `requests`, HTTP Methods & Status Codes, API Authentication, Boto3 (AWS SDK) Intro

📌 Key Concepts Overview

Concept	One-Line Definition
API	A way for two applications/systems to communicate over HTTP
requests	Python library for making HTTP requests (GET, POST, etc.)
HTTP Method	The "verb" describing what action you want (GET, POST, PUT, PATCH, DELETE)
Status Code	A number the server returns indicating success/failure of a request
response.json()	Parses the response body as a Python dict/list
Authentication Token	A secret credential used to prove identity to an API
boto3	The official AWS SDK for Python — used to automate AWS resources
boto3.client()	Creates a low-level client to interact with a specific AWS service

---

🌐 Part 1 — What Is an API?

API = Application Programming Interface — a defined way for one piece of software to talk to another. In DevOps, you'll constantly interact with APIs: AWS APIs, GitHub APIs, Slack webhooks, monitoring tool APIs (Datadog, PagerDuty), CI/CD tool APIs (Jenkins, GitLab).

import requests

The requests library is the standard, most widely used Python tool for making HTTP API calls.

---

📬 Part 2 — HTTP Methods

Method	Purpose	DevOps Example
GET	Fetch/read data	Get list of EC2 instances, check API health
POST	Send/create new data	Create a new resource, trigger a deployment
PUT	Update FULL data (replace entirely)	Replace an entire config object
PATCH	Update PARTIAL data (only changed fields)	Update just one field in a config
DELETE	Remove data	Delete a resource, terminate an instance

# Basic GET request
import requests
URL = "https://www.python.org/"
response = requests.get(URL)
print(response)               # <Response [200]>
print(response.status_code)    # 200
print(response.text)            # raw HTML/text content of the response

---

🔢 Part 3 — HTTP Status Codes

Code	Meaning	Category
200	SUCCESS	2xx — Success
201	CREATED	2xx — Success
400	BAD REQUEST	4xx — Client Error
401	UNAUTHORIZED	4xx — Client Error (missing/invalid auth)
403	FORBIDDEN	4xx — Client Error (no permission)
404	NOT FOUND	4xx — Client Error
415	UNSUPPORTED MEDIA	4xx — Client Error
500	SERVER ERROR	5xx — Server Error

Quick mental model:

• 2xx = it worked
• 4xx = YOU (the client) did something wrong — bad input, missing auth, wrong URL
• 5xx = THE SERVER broke — not your fault

# Always check status_code before trusting the response
if response.status_code == 200:
    print('✅ Request succeeded')
elif response.status_code == 401:
    print('❌ Authentication failed — check your token')
elif response.status_code == 404:
    print('❌ Resource not found')
else:
    print(f'⚠️ Unexpected status: {response.status_code}')

---

📦 Part 4 — Working with the Response Object

# .json() — parses response body as a Python dict/list (when API returns JSON)
data = response.json()
print(data)

# .headers — metadata about the response (server, content-type, rate limits, etc.)
print(response.headers)
print(response.headers.get("Content-Type"))    # get a SPECIFIC header value

response.text vs response.json():

Property/Method	Returns	Use when
response.text	Raw string	The response is HTML or plain text
response.json()	Python dict/list	The response is JSON (most modern APIs)

---

🔐 Part 5 — API Authentication

Most real-world APIs require authentication — you can't just call them anonymously like python.org.

Using getpass to Securely Capture a Token

from requests.auth import HTTPBasicAuth
import getpass

url = "https://api.github.com/user"
username = "your_github_username"
my_token = getpass.getpass("Enter your github token: ")   # hides input — doesn't echo to screen

headers = {"Authorization": f"Bearer {my_token}"}
response = requests.get(url, headers=headers)
print(response.status_code)
print(response.json())

Why getpass.getpass() instead of input()? It doesn't display what you type on screen — critical for entering tokens/passwords so they're not visible to anyone looking at your screen or captured in a saved notebook output.

The Authorization Header Pattern

import requests

token = "your_github_personal_access_token"

r = requests.get(
    "https://api.github.com/user",
    headers={"Authorization": f"Bearer {token}"}
)

print(r.json())

⚠️ CRITICAL SECURITY WARNING:

The class notebook had a real-looking GitHub token hardcoded directly in the code (token = "ghp_..."). Never do this in real work or in anything you save/commit/share — including Dev.to posts or GitHub repos. Hardcoded tokens in notebooks or scripts are one of the most common ways credentials leak publicly. If a token like this is ever committed to GitHub or shared anywhere public, revoke it immediately in GitHub Settings → Developer Settings → Personal Access Tokens, and generate a new one.

Best practice instead:

import os
token = os.environ.get('GITHUB_TOKEN')     # read from environment variable
# or use getpass.getpass() for interactive scripts, as shown above

---

🕸️ Part 6 — Web Content Fetching + Regex Extraction

import urllib.request
from re import findall

url = "https://www.summet.com/dmsi/html/codesamples/addresses.html"
response = urllib.request.urlopen(url)
html = response.read()
print(html, type(html))      # raw bytes, <class 'bytes'>

print(html.decode())           # decode bytes → readable string

Extracting Phone Numbers with Regex (from class)

# Pattern to extract US-style phone numbers like [phone]
data = findall(r'\(\d{3}\)\s\d{3}-\d{4}', html.decode())
for i in data:
    print(i)

Working pattern is:

r'\(\d{3}\)\s\d{3}-\d{4}'
# \(        — literal opening parenthesis
# \d{3}     — exactly 3 digits (area code)
# \)        — literal closing parenthesis
# \s        — one whitespace character
# \d{3}     — exactly 3 digits
# -         — literal hyphen
# \d{4}     — exactly 4 digits

DevOps relevance: This combination — fetching a URL's content and extracting structured data with regex — is exactly how you'd scrape status pages, parse third-party documentation for config values, or pull structured info out of HTML-based dashboards that don't offer a clean JSON API.

---

☁️ Part 7 — Boto3 (AWS SDK for Python) — Introduction

Boto3 is the official Python library/SDK for automating AWS. This is the single most important library for your DevOps/Cloud career — almost all AWS automation in Python goes through Boto3.

import boto3
from pprint import pprint     # pretty-print — formats nested dicts/lists readably

iam_client = boto3.client('iam')      # create a "client" for the IAM service
iam_client.list_users()                  # returns a dict of all IAM users
pprint(iam_client.list_users())           # same data, but readable/formatted

The boto3.client() Pattern

# Syntax: boto3.client('service_name')
iam_client = boto3.client('iam')
ec2_client = boto3.client('ec2')      # (not in class yet, but same pattern)
s3_client = boto3.client('s3')         # (same pattern, different service)

This is the foundational pattern for ALL of Boto3: create a client for a service, then call methods on it that map to AWS API actions.

Looping Through API Response Data

import boto3
iam_client = boto3.client('iam')
users = iam_client.list_users()

for i in users['Users']:           # 'Users' key holds a LIST of user dicts
    print(i['UserName'])             # each user is a dict — access fields by key

This is a direct continuation of the dict/list knowledge from earlier classes — AWS API responses are nested Python dicts and lists, exactly like the structures practiced in Class 2 and Class 6.

Creating Resources (Preview — Incomplete in Class)

# Creating an IAM user (class left this incomplete — syntax shown for reference)
import boto3
iam_client = boto3.client('iam')
iam_client.create_user(UserName='new-devops-user')    # correct method name is create_user (singular)

Note: The notebook had iam_client.create_users() (plural) which is not a valid Boto3 method — the correct method is create_user() (singular), which requires at minimum a UserName parameter. This was likely a class typo/work-in-progress — worth double-checking against the Boto3 IAM docs when you actually use it.

---

☁️ DevOps / Cloud Use Cases

# 1. Health check script using requests
import requests

def check_service_health(url):
    try:
        response = requests.get(url, timeout=5)
        if response.status_code == 200:
            print(f'✅ {url} is healthy')
        else:
            print(f'⚠️ {url} returned {response.status_code}')
    except requests.exceptions.RequestException as e:
        print(f'❌ {url} unreachable: {e}')

check_service_health('https://api.github.com')

# 2. Trigger a CI/CD pipeline via API (POST pattern)
import requests
import os

token = os.environ.get('CI_TOKEN')
headers = {'Authorization': f'Bearer {token}'}
payload = {'branch': 'main', 'environment': 'staging'}

# response = requests.post('https://ci.example.com/api/trigger', headers=headers, json=payload)

# 3. List all running EC2 instances with Boto3 (preview of next classes)
import boto3
ec2_client = boto3.client('ec2')
# instances = ec2_client.describe_instances()
# for reservation in instances['Reservations']:
#     for instance in reservation['Instances']:
#         print(instance['InstanceId'], instance['State']['Name'])

# 4. Audit all IAM users (directly from class)
import boto3
iam_client = boto3.client('iam')
users = iam_client.list_users()
for user in users['Users']:
    print(f"User: {user['UserName']} | Created: {user['CreateDate']}")

# 5. Securely load a GitHub token from environment instead of hardcoding
import os
import requests

token = os.environ.get('GITHUB_TOKEN')      # ✅ never hardcode tokens
headers = {'Authorization': f'Bearer {token}'}
response = requests.get('https://api.github.com/user', headers=headers)
print(response.status_code)

---

❌ Common Mistakes

Mistake	Issue	Fix
Hardcoding tokens/secrets in code	Massive security risk — gets leaked via Git history, notebooks, screenshots	Use os.environ.get() or getpass.getpass()
Not checking status_code before using .json()	Calling .json() on a failed/non-JSON response → crash	Check response.status_code == 200 first
Using response.text for JSON APIs	Gives raw string, not a usable dict	Use response.json() for JSON responses
iam_client.create_users() (plural)	Not a real Boto3 method	Correct method is create_user() (singular)
Forgetting headers dict for authenticated requests	Gets 401 Unauthorized	Always pass headers={"Authorization": f"Bearer {token}"}
Typo'd regex with unescaped \d	d\{4} instead of \d{4} — silently matches wrong things or errors	Double-check regex syntax — \d{4} not d\{4}
Not setting a timeout on requests.get()	Script can hang forever on an unresponsive server	Always pass timeout=5 (or similar)

---

🎯 Interview Points

1. "What's the difference between GET, POST, PUT, and PATCH?"
   → GET retrieves data without modifying it. POST creates a new resource. PUT replaces an entire resource. PATCH updates only specific fields of a resource.

2. "What does a 401 vs 403 status code mean?"
   → 401 Unauthorized means you're not authenticated at all (missing/invalid credentials). 403 Forbidden means you ARE authenticated, but don't have permission for that specific action.

3. "Why should you never hardcode API tokens in your code?"
   → Hardcoded secrets can be leaked through version control history, shared notebooks, or screenshots — and unlike a password, anyone who obtains the token can impersonate your account/service until it's revoked.

4. "What's the difference between response.text and response.json()?"
   → response.text returns the raw response body as a string. response.json() parses that body as JSON and returns a Python dict/list — but only works if the response is actually valid JSON.

5. "What is Boto3 and how does it work at a high level?"
   → Boto3 is AWS's official Python SDK. You create a client for a specific service (boto3.client('iam'), boto3.client('ec2')), then call methods on that client that correspond to AWS API actions — responses come back as Python dicts/lists.

6. "How would you securely pass a secret/token in a Python script?"
   → Use environment variables (os.environ.get('TOKEN')) for automated/scheduled scripts, or getpass.getpass() for interactive scripts — never hardcode it directly in source code.

---

📚 Knowledge Base — Quick Revision

# ── REQUESTS BASICS ─────────────────────────────────────────
import requests
response = requests.get(url)               # GET request
response = requests.post(url, json=data)    # POST request

response.status_code        # e.g. 200, 404, 500
response.text                 # raw string body
response.json()                # parsed dict/list (if JSON)
response.headers                # response metadata dict

# ── HTTP METHODS ────────────────────────────────────────────
GET     # fetch
POST    # create
PUT     # full update
PATCH   # partial update
DELETE  # remove

# ── STATUS CODES ────────────────────────────────────────────
200 OK | 201 Created | 400 Bad Request | 401 Unauthorized
403 Forbidden | 404 Not Found | 500 Server Error

# ── AUTHENTICATION ──────────────────────────────────────────
import getpass
token = getpass.getpass('Enter token: ')        # hidden input
headers = {"Authorization": f"Bearer {token}"}
response = requests.get(url, headers=headers)

# ✅ secure: read from environment
import os
token = os.environ.get('MY_TOKEN')

# ── BOTO3 (AWS SDK) ──────────────────────────────────────────
import boto3
client = boto3.client('service_name')    # e.g. 'iam', 'ec2', 's3'
client.list_users()                        # example IAM call
client.create_user(UserName='name')         # correct method (singular)

from pprint import pprint
pprint(response_dict)                          # readable nested output

---

🏋️ Practice Questions

Easy

1. Make a GET request to https://api.github.com (no auth needed) and print the status code and the Content-Type header.
2. What is the difference between a 404 and a 500 status code? Give a real-world example of each.
3. Use getpass.getpass() to securely capture a fake "password" from the user and print its length (not the value itself).

Medium

1. Write a function check_status(url) that makes a GET request, and based on the status code, prints "Healthy", "Not Found", "Unauthorized", or "Server Error" accordingly. Use try-except to catch connection failures too.
2. Make an authenticated GET request to https://api.github.com/user using a personal access token read from an environment variable (os.environ.get('GITHUB_TOKEN')) rather than hardcoded. Print just the login field from the JSON response.
3. Use boto3.client('iam') to list all IAM users in your AWS account, and print each username along with their CreateDate.

DevOps-Focused

1. Multi-Endpoint Health Checker: Write a script that takes a list of URLs (e.g., ['https://api.github.com', 'https://www.python.org']), loops through them, makes a GET request to each with a 5-second timeout, and prints a health summary table (URL → status code → Healthy/Unhealthy). Wrap each request in try-except so one failure doesn't stop the loop.
2. IAM User Auditor: Using boto3.client('iam'), write a script that lists all IAM users and flags (prints a warning for) any user whose CreateDate is more than 365 days ago — a common security audit task (old, possibly unused credentials).

---