Wenlo

Posted on Nov 10 Edited on Nov 10 AI-assisted

Find failing endpoints and slow requests with Python on iPhone

#python #beginners #debugging #tutorial

A slow request and a server error are different clues. This small log detective answers three questions: which endpoints return server failures, how slow a typical request is, and which individual requests deserve a closer look.

The data is fictional and embedded in the script. You do not need a server, a real access log, or any network connection. The example uses only Python's standard library.

1. Define the format before parsing

Each valid line contains a method, path, three-digit status, and an integer duration ending in ms. re.fullmatch requires the entire line to fit. A partial match could hide unexpected text at the end of a line.

This is a deliberately simplified format, not a general parser for Nginx, Apache, or arbitrary application logs. It accepts GET and POST, and its paths exclude query strings and many other valid URL characters. Rejected line numbers remain visible so that bad input does not silently disappear.

2. Give the error rate a denominator

For this report, a server failure means an HTTP status from 500 through 599. The 404 row is included in the request count but is not a server failure. Three of the ten parsed requests are 5xx responses, giving a 30% server-error rate.

The malformed seventh line is excluded from that denominator. If many lines are malformed, the rate can be misleading even if the arithmetic is correct. Always read the rejection count alongside the percentage.

3. Use two views of latency

The median describes the middle of the sorted latency values and is less sensitive to a few slow requests than the mean. With ten observations, it averages the fifth and sixth values: (180 + 210) / 2 = 195 ms.

The slowest three requests answer another question: which specific records should we inspect? Sorting retains the path, status, and duration together so the output still has context.

Run the complete example

Create log_detective.py in PythonIDE and run this code:

import re
from collections import Counter
from statistics import median

# Fictional, simplified access logs. No real users.
logs = '''GET /home 200 42ms
GET /search 200 180ms
POST /checkout 500 920ms
GET /home 200 38ms
GET /search 503 1100ms
POST /checkout 200 260ms
broken log line
GET /product 200 75ms
POST /checkout 500 870ms
GET /search 200 210ms
GET /missing 404 25ms'''

# Accept the whole line, not a partial match.
pattern = re.compile(
    r'(GET|POST) (/[\w/-]*) (\d{3}) (\d+)ms'
)
requests, rejected = [], []
for number, line in enumerate(logs.splitlines(), 1):
    match = pattern.fullmatch(line)
    if not match:
        rejected.append(number)
        continue
    method, path, status, ms = match.groups()
    requests.append((method, path, int(status), int(ms)))

# 5xx means a server failure; 404 is not a 5xx.
failures = [r for r in requests if 500 <= r[2] < 600]
counts = Counter(r[1] for r in failures)
slowest = sorted(requests, key=lambda r: r[3], reverse=True)
latencies = [r[3] for r in requests]

print('LOG DETECTIVE: FICTIONAL DATA')
print(f'Parsed: {len(requests)}; rejected lines: {rejected}')
print(f'Server error rate: {len(failures)/len(requests):.0%}')
print(f'Median latency: {median(latencies):.0f} ms')
print('\n5xx errors by endpoint:')
for path, count in counts.most_common():
    print(f'  {path}: {count}')
print('\nThree slowest requests:')
for method, path, status, ms in slowest[:3]:
    print(f'  {method} {path}: {status}, {ms} ms')
print('\nNext step: inspect checkout failures.')
print('A log summary suggests where to look, not a cause.')

Check the output

Running this exact sample in PythonIDE produced:

Parsed requests: 10. Rejected line: 7.
Server-error rate: 30%.
Median latency: 195 ms.
5xx counts: /checkout has 2; /search has 1.
Slowest requests: /search at 1100 ms, /checkout at 920 ms, and /checkout at 870 ms.

Checkout is a reasonable place to start investigating because it appears twice in the failure list. That does not identify the cause. A real investigation would connect these records to request IDs, timestamps, traces, and application errors before concluding that a dependency or code change is responsible.

Know the boundary

The sample deliberately contains valid requests. If the input is empty or every line is rejected, dividing by len(requests) fails and median has no data. Before adapting this to real input, add a guard that prints “No valid requests” and skips the aggregate calculations when requests is empty.

The regular expression checks shape, not every HTTP semantic rule. It also accepts three-digit values outside the usual HTTP status range. A production parser should validate that range, document duration units, and handle its actual log format.

Try one change

Change the 404 to a 500 and predict the new server-error rate before running. Then add a valid, very slow 200 response. Does it change the server-error count, the denominator, the median, or the top-three list?

I build PythonIDE, a Python coding app for iPhone and iPad. The parsing and reporting ideas here also work in other Python environments.

View PythonIDE on the App Store

AI assisted with drafting. The example was run and its output checked in PythonIDE.

DEV Community