API Design Mastery: From Engineer to Architect

FOUNDATION: What Makes a Good API?

The Core Philosophy

An API is a contract. It's not just code — it's a promise to every developer who builds on top of it. A broken promise costs companies millions in migration costs, lost trust, and engineering hours. Before writing a single endpoint, understand this:

A good API is designed for its consumers, not its implementers.

The best APIs feel like they were designed by someone who deeply understood the consumer's problem, not someone who just exposed their database schema.

The Five Pillars of API Excellence

1. Consistency

Every decision you make must be applied uniformly. If you use snake_case for one field, it must be snake_case everywhere. Inconsistency is the number one source of developer frustration and bugs.

2. Predictability

A developer who learns one part of your API should be able to predict how another part works. Stripe's API is famous for this — once you understand how create works for one resource, you know how it works for all of them.

3. Discoverability

The API should guide developers toward correct usage. Good naming, good documentation, and good error messages reduce the need to read docs at all.

4. Stability / Backward Compatibility

An API that breaks its consumers with every release is worse than no API. Stability earns trust. Stripe's API has endpoints from 2011 that still work today.

5. Security by Default

An API that requires developers to "opt into" security is an insecure API. Every endpoint must be secure unless explicitly made public, not the other way around.

---

Why It Matters in Real-World Systems

In 2012, Amazon's Jeff Bezos issued the famous API mandate: every team must expose their functionality through APIs, all communication must happen through those APIs, and no exceptions. This turned Amazon into AWS — a $90B+ business built entirely on API design decisions made 20 years ago.

Bad API design compounds over time:

• Every bad decision becomes harder to fix as consumers grow
• Breaking changes require deprecation periods, migration guides, versioning overhead
• Poor error messages generate support tickets that cost real money
• Security flaws in APIs are the #1 attack surface in modern systems (OWASP API Top 10)

---

What Companies Do

Stripe: Obsesses over developer experience. Their API design team reviews every endpoint before it ships. They've maintained backward compatibility for 13+ years. They test their API by making their own engineers build with it before shipping.

GitHub: Uses REST and GraphQL side-by-side. They moved to GraphQL (v4) because REST wasn't expressive enough for their complex relationship graphs, but kept REST (v3) alive for simplicity.

Google: Follows the Google API Design Guide (aip.dev), which is a masterclass in resource-oriented design. They use proto-first design — define the contract in Protocol Buffers before writing any implementation.

Amazon (AWS): Prioritizes backward compatibility above almost everything else. The S3 API from 2006 still works identically today. They version at the service level, not the endpoint level.

---

MODULE 1: Resource-Oriented API Design

The Concept

REST is fundamentally about resources, not actions. A resource is a noun — a thing that exists in your system. Every URL should identify a resource, and HTTP methods express what you want to do with it.

The single biggest mistake engineers make: thinking in terms of procedures instead of resources.

Procedural thinking (wrong):

/getUser
/createUser
/deleteUser
/getUserOrders
/cancelOrder

Resource thinking (correct):

/users          → collection of users
/users/{id}     → a specific user
/users/{id}/orders  → orders belonging to a user
/orders/{id}    → a specific order

The HTTP method is the verb. You don't need it in the URL.

---

REST Constraints (The Real REST, Not the Popular Myth)

Roy Fielding defined REST with 6 constraints in his 2000 dissertation. Most "REST APIs" violate several of these. Understanding them separates engineers from architects:

1. Client-Server Separation

The client and server are independent. The server doesn't care about the UI. The client doesn't know about the database. This enables them to evolve independently.

2. Statelessness (most violated)

Every request must contain all information necessary to process it. The server stores no session state between requests. Authentication tokens, pagination cursors, request context — all must come from the client on every request.

// WRONG: Server maintains session
POST /login          → server stores "user is authenticated"
GET /profile         → server checks its own session

// RIGHT: Stateless
GET /profile
Authorization: Bearer eyJhbGci...  ← client sends auth on every request

3. Cacheability

Responses must declare whether they can be cached. HTTP has a rich caching model (Cache-Control, ETag, Last-Modified) — use it.

4. Uniform Interface (the core constraint)

All resources are accessed through a consistent interface. This is what makes REST learnable.

5. Layered System

The client doesn't know if it's talking to the origin server, a load balancer, a CDN, or an API gateway. Each layer can only see the layer adjacent to it.

6. Code on Demand (optional)

Servers can send executable code (JavaScript) to clients. Rarely used in APIs.

---

URI Naming Conventions

Rules

Rule	Bad	Good
Use nouns, not verbs	/getUsers	/users
Use plural for collections	/user	/users
Use lowercase	/Users	/users
Use hyphens, not underscores	/user_orders	/user-orders
No file extensions	/users.json	/users
No trailing slash	/users/	/users
Hierarchy = relationship	/getOrdersForUser/5	/users/5/orders

Bad Examples

GET /getAllActiveUsers
GET /user/getById/5
POST /createNewOrder
DELETE /removeUserAccount/5
GET /users/5/getOrderHistory
POST /updateUserProfile

Good Examples

GET /users?status=active
GET /users/5
POST /orders
DELETE /users/5
GET /users/5/orders
PATCH /users/5

Nested Resource Depth

A common mistake is nesting too deeply:

// BAD: Too deep, fragile
GET /companies/5/departments/3/teams/8/employees/12/tasks/4

// GOOD: Max 2 levels of nesting, then use query params or direct resource
GET /tasks/4
GET /employees/12/tasks

Rule of thumb: Never go deeper than 2 levels of nesting. If you need more context, use query parameters or rethink your resource model.

---

HTTP Methods — The Complete Picture

GET

• Purpose: Retrieve a resource or collection. Never modify state.
• Idempotent: Yes. Calling GET 100 times produces the same result.
• Safe: Yes. Has no side effects.
• Body: No request body (technically allowed but universally avoided).
• Caching: Must be cacheable.

GET /users/5 HTTP/1.1
Accept: application/json
Authorization: Bearer token

HTTP/1.1 200 OK
Content-Type: application/json
Cache-Control: max-age=60

{
  "id": 5,
  "name": "Ali Al-Ghamdi",
  "email": "[e-mail]"
}

POST

• Purpose: Create a new resource. The server assigns the ID.
• Idempotent: No. Calling POST twice creates two resources.
• Safe: No.
• Response: 201 Created with Location header pointing to new resource.

POST /users HTTP/1.1
Content-Type: application/json

{
  "name": "Ali Al-Ghamdi",
  "email": "[e-mail]",
  "password": "secure_password"
}

HTTP/1.1 201 Created
Location: /users/5
Content-Type: application/json

{
  "id": 5,
  "name": "Ali Al-Ghamdi",
  "email": "[e-mail]",
  "created_at": "2026-06-06T10:00:00Z"
}

PUT

• Purpose: Full replacement of a resource. If the resource doesn't exist, create it (some implementations). Client provides the complete resource.
• Idempotent: Yes. PUT with the same body always produces the same result.
• Key distinction from PATCH: PUT replaces the entire resource. Missing fields are set to null/default.

PUT /users/5 HTTP/1.1
Content-Type: application/json

{
  "name": "Ali Al-Ghamdi Updated",
  "email": "[e-mail]",
  "phone": "[phone]"
}

HTTP/1.1 200 OK

PATCH

• Purpose: Partial update. Only send the fields you want to change.
• Idempotent: Depends on implementation. Should be designed to be idempotent.
• Key principle: Fields not included in the request body are left unchanged.

PATCH /users/5 HTTP/1.1
Content-Type: application/json

{
  "phone": "[phone]"
}

HTTP/1.1 200 OK
Content-Type: application/json

{
  "id": 5,
  "name": "Ali Al-Ghamdi",
  "email": "[e-mail]",
  "phone": "[phone]",
  "updated_at": "2026-06-06T10:05:00Z"
}

DELETE

• Purpose: Remove a resource.
• Idempotent: Yes. Deleting a resource that doesn't exist should return 404 on the first call, but architecturally should not error on repeat calls (return 204 or 404 consistently — pick one and document it).
• Soft vs Hard delete: Most production systems do soft delete (mark as deleted, keep the record).

DELETE /users/5 HTTP/1.1
Authorization: Bearer token

HTTP/1.1 204 No Content

Method Override (Legacy Systems)

Some firewalls block PUT/PATCH/DELETE. Support method override via header:

POST /users/5 HTTP/1.1
X-HTTP-Method-Override: PATCH
Content-Type: application/json

---

Idempotency — The Most Underrated Concept

The Concept

An operation is idempotent if performing it multiple times produces the same result as performing it once.

f(f(x)) = f(x)

This is critical in distributed systems where:

• Networks fail mid-request
• Clients retry on timeout
• Message queues deliver at-least-once

Why It Matters

Imagine a payment API. The client sends a payment request, the network drops, and the client never gets a response. Did the payment go through? If the client retries without idempotency protection, the customer gets charged twice. This is a catastrophic bug.

Implementation: Idempotency Keys

Stripe pioneered this pattern, now industry standard:

POST /payments HTTP/1.1
Idempotency-Key: 8f14e45f-ceea-467a-a866-77776f28b186
Content-Type: application/json

{
  "amount": 10000,
  "currency": "SAR",
  "customer_id": "cust_123"
}

Server behavior:

1. Check if Idempotency-Key has been seen before
2. If not: process the request, store the result with the key
3. If yes: return the stored result immediately without reprocessing
4. Store idempotency keys for 24 hours (Stripe) or configurable period

Laravel Implementation

// Middleware: IdempotencyMiddleware.php
class IdempotencyMiddleware
{
    public function handle(Request $request, Closure $next): Response
    {
        if (!in_array($request->method(), ['POST', 'PATCH'])) {
            return $next($request);
        }

        $idempotencyKey = $request->header('Idempotency-Key');

        if (!$idempotencyKey) {
            // For payment/critical endpoints, require it
            // For non-critical, just pass through
            return $next($request);
        }

        $cacheKey = "idempotency:{$request->user()->id}:{$idempotencyKey}";

        if (Cache::has($cacheKey)) {
            $cachedResponse = Cache::get($cacheKey);
            return response()->json(
                $cachedResponse['body'],
                $cachedResponse['status']
            )->header('Idempotency-Replay', 'true');
        }

        $response = $next($request);

        // Only cache successful responses
        if ($response->getStatusCode() < 500) {
            Cache::put($cacheKey, [
                'body' => json_decode($response->getContent(), true),
                'status' => $response->getStatusCode(),
            ], now()->addHours(24));
        }

        return $response;
    }
}

Bad Example (No Idempotency)

// Client sends payment
POST /payments
{ "amount": 100 }
// Network timeout — client retries

POST /payments
{ "amount": 100 }
// Result: customer charged twice

Good Example (With Idempotency Key)

POST /payments
Idempotency-Key: uuid-12345
{ "amount": 100 }
// Network timeout — client retries with SAME key

POST /payments
Idempotency-Key: uuid-12345
{ "amount": 100 }
// Server returns cached result — customer charged once

---

HTTP Status Codes — The Complete Guide

The Categories

Range	Category	Usage
1xx	Informational	Rarely used in REST APIs
2xx	Success	Request was received, understood, processed
3xx	Redirection	Resource moved, use another URL
4xx	Client Error	The client did something wrong
5xx	Server Error	We (the server) did something wrong

Essential Status Codes

2xx Success:

Code	Name	When to Use
200	OK	GET, PUT, PATCH success with body
201	Created	POST success, new resource created
202	Accepted	Async operation started, not yet complete
204	No Content	DELETE success, or PUT/PATCH with no response body
206	Partial Content	Partial GET (range requests, chunked transfers)

4xx Client Errors:

Code	Name	When to Use
400	Bad Request	Malformed syntax, invalid data
401	Unauthorized	Not authenticated (missing/invalid token)
403	Forbidden	Authenticated but not authorized
404	Not Found	Resource doesn't exist
405	Method Not Allowed	GET on a POST-only endpoint
409	Conflict	State conflict (duplicate email, concurrent edit)
410	Gone	Resource permanently deleted (stronger than 404)
422	Unprocessable Entity	Validation errors (syntactically valid but semantically wrong)
429	Too Many Requests	Rate limit exceeded

5xx Server Errors:

Code	Name	When to Use
500	Internal Server Error	Unexpected server error
502	Bad Gateway	Upstream service failed
503	Service Unavailable	Server overloaded or in maintenance
504	Gateway Timeout	Upstream service timed out

The 401 vs 403 Distinction

This trips up almost every engineer:

401 Unauthorized = "I don't know who you are" (Authentication failure)
403 Forbidden    = "I know who you are, but you can't do this" (Authorization failure)

// 401: No token provided
GET /orders/5
HTTP/1.1 401 Unauthorized
WWW-Authenticate: Bearer realm="api"

// 403: Token valid, but user doesn't own this order
GET /orders/999 (belongs to different user)
HTTP/1.1 403 Forbidden

Security note: Sometimes you should return 404 instead of 403. If a user shouldn't know a resource exists (e.g., another user's private data), return 404. This prevents information disclosure — an attacker can't enumerate resources they can't access.

---

Request Validation

The Principle

Validate at the boundary. Every API endpoint is a trust boundary — treat all incoming data as potentially malicious. Validate before it touches your business logic or database.

Validation Layers

Client Request
    ↓
Schema Validation   (correct types, required fields, format)
    ↓
Business Validation (unique email, valid foreign key, logical constraints)
    ↓
Authorization Check (can this user do this to this resource?)
    ↓
Business Logic

Bad Error Response

{
  "error": "Validation failed"
}

This tells the developer nothing actionable.

Good Error Response (RFC 7807 - Problem Details)

{
  "type": "https://api.example.com/errors/validation",
  "title": "Validation Failed",
  "status": 422,
  "detail": "The request contains invalid fields",
  "instance": "/users",
  "errors": {
    "email": [
      "The email field is required.",
      "The email must be a valid email address."
    ],
    "password": [
      "The password must be at least 8 characters."
    ],
    "phone": [
      "The phone must match the format +[country_code][number]."
    ]
  },
  "trace_id": "7f3a9b2c-1234-5678-abcd-ef0123456789"
}

Laravel Implementation

// app/Http/Requests/CreateUserRequest.php
class CreateUserRequest extends FormRequest
{
    public function authorize(): bool
    {
        return true; // Authorization handled separately
    }

    public function rules(): array
    {
        return [
            'name'     => ['required', 'string', 'max:255'],
            'email'    => ['required', 'email:rfc,dns', 'unique:users,email'],
            'password' => ['required', 'string', 'min:8', 'confirmed'],
            'phone'    => ['nullable', 'string', 'regex:/^\+[1-9]\d{7,14}$/'],
            'role'     => ['required', Rule::in(['admin', 'user', 'viewer'])],
        ];
    }

    public function messages(): array
    {
        return [
            'email.unique' => 'An account with this email already exists.',
            'phone.regex'  => 'Phone must include country code (e.g., [phone]).',
        ];
    }

    protected function failedValidation(Validator $validator): void
    {
        throw new HttpResponseException(
            response()->json([
                'type'     => 'https://api.example.com/errors/validation',
                'title'    => 'Validation Failed',
                'status'   => 422,
                'detail'   => 'One or more fields are invalid.',
                'errors'   => $validator->errors(),
                'trace_id' => request()->header('X-Request-ID', Str::uuid()),
            ], 422)
        );
    }
}

---

Response Formatting

The Envelope Debate

There are two schools of thought:

No envelope (GitHub, Stripe use this for individual resources):

{
  "id": 5,
  "name": "Ali Al-Ghamdi",
  "email": "[e-mail]"
}

With envelope (common for consistency and metadata):

{
  "data": {
    "id": 5,
    "name": "Ali Al-Ghamdi",
    "email": "[e-mail]"
  },
  "meta": {
    "request_id": "abc-123",
    "timestamp": "2026-06-06T10:00:00Z"
  }
}

My recommendation for production systems: Use a consistent envelope for all responses. The metadata becomes invaluable for debugging and observability.

Collection Response Format

{
  "data": [
    { "id": 1, "name": "User One" },
    { "id": 2, "name": "User Two" }
  ],
  "meta": {
    "current_page": 1,
    "per_page": 20,
    "total": 243,
    "total_pages": 13,
    "from": 1,
    "to": 20
  },
  "links": {
    "self":  "/users?page=1&per_page=20",
    "first": "/users?page=1&per_page=20",
    "prev":  null,
    "next":  "/users?page=2&per_page=20",
    "last":  "/users?page=13&per_page=20"
  }
}

Error Response Format

{
  "error": {
    "type":    "https://api.example.com/errors/not-found",
    "title":   "Resource Not Found",
    "status":  404,
    "detail":  "No user found with ID 999",
    "instance": "/users/999",
    "trace_id": "7f3a9b2c-1234-5678",
    "timestamp": "2026-06-06T10:00:00Z",
    "docs": "https://docs.example.com/errors/not-found"
  }
}

Laravel API Resource Implementation

// app/Http/Resources/UserResource.php
class UserResource extends JsonResource
{
    public function toArray(Request $request): array
    {
        return [
            'id'         => $this->id,
            'name'       => $this->name,
            'email'      => $this->email,
            'phone'      => $this->phone,
            'role'       => $this->role,
            'status'     => $this->status,
            'created_at' => $this->created_at->toISOString(),
            'updated_at' => $this->updated_at->toISOString(),

            // Conditional fields — only include when loaded
            'orders_count' => $this->when(
                $this->relationLoaded('orders'),
                fn() => $this->orders->count()
            ),

            // Conditional relationships
            'orders' => OrderResource::collection(
                $this->whenLoaded('orders')
            ),

            // Conditional for admin only
            'internal_notes' => $this->when(
                $request->user()?->isAdmin(),
                $this->internal_notes
            ),
        ];
    }
}

// app/Http/Resources/UserCollection.php
class UserCollection extends ResourceCollection
{
    public function toArray(Request $request): array
    {
        return [
            'data'  => $this->collection,
            'meta'  => [
                'total'        => $this->total(),
                'per_page'     => $this->perPage(),
                'current_page' => $this->currentPage(),
                'total_pages'  => $this->lastPage(),
                'from'         => $this->firstItem(),
                'to'           => $this->lastItem(),
            ],
            'links' => [
                'self'  => $this->url($this->currentPage()),
                'first' => $this->url(1),
                'prev'  => $this->previousPageUrl(),
                'next'  => $this->nextPageUrl(),
                'last'  => $this->url($this->lastPage()),
            ],
        ];
    }
}

---

MODULE 2: Error Handling Standards

The Principle

Errors are part of your API's contract. A great API makes errors as useful as its successes. Your error responses should answer:

1. What went wrong (machine-readable code)
2. Why it went wrong (human-readable message)
3. Where it went wrong (which field, which resource)
4. How to fix it (documentation link, suggestion)
5. How to track it (trace ID for support)

Laravel Global Exception Handler

// app/Exceptions/Handler.php
class Handler extends ExceptionHandler
{
    public function render($request, Throwable $e): Response
    {
        if ($request->expectsJson()) {
            return $this->renderApiException($request, $e);
        }

        return parent::render($request, $e);
    }

    private function renderApiException(Request $request, Throwable $e): JsonResponse
    {
        $traceId = $request->header('X-Request-ID', Str::uuid()->toString());

        return match(true) {
            $e instanceof ValidationException => response()->json([
                'error' => [
                    'type'     => 'validation_error',
                    'title'    => 'Validation Failed',
                    'status'   => 422,
                    'errors'   => $e->errors(),
                    'trace_id' => $traceId,
                ]
            ], 422),

            $e instanceof ModelNotFoundException => response()->json([
                'error' => [
                    'type'    => 'not_found',
                    'title'   => 'Resource Not Found',
                    'status'  => 404,
                    'detail'  => "The requested resource does not exist.",
                    'trace_id' => $traceId,
                ]
            ], 404),

            $e instanceof AuthorizationException => response()->json([
                'error' => [
                    'type'    => 'forbidden',
                    'title'   => 'Access Denied',
                    'status'  => 403,
                    'detail'  => 'You do not have permission to perform this action.',
                    'trace_id' => $traceId,
                ]
            ], 403),

            $e instanceof ThrottleRequestsException => response()->json([
                'error' => [
                    'type'       => 'rate_limit_exceeded',
                    'title'      => 'Too Many Requests',
                    'status'     => 429,
                    'detail'     => 'You have exceeded the request limit. Please wait before retrying.',
                    'retry_after' => $e->getHeaders()['Retry-After'] ?? 60,
                    'trace_id'   => $traceId,
                ]
            ], 429),

            default => response()->json([
                'error' => [
                    'type'    => 'internal_error',
                    'title'   => 'Internal Server Error',
                    'status'  => 500,
                    'detail'  => config('app.debug')
                        ? $e->getMessage()
                        : 'An unexpected error occurred. Please try again later.',
                    'trace_id' => $traceId,
                ]
            ], 500),
        };
    }
}

Critical note: Never leak stack traces, SQL queries, file paths, or internal service names in production error responses. config('app.debug') must be false in production.

---

MODULE 3: API Versioning

The Problem

APIs change. Features are added, fields are renamed, endpoints are restructured. But once consumers exist, you can't simply break them. Versioning gives you a way to evolve while maintaining backward compatibility.

Versioning Strategies

Strategy 1: URL Path Versioning (most common)

/v1/users
/v2/users

Pros: Obvious, easy to test in browser, easy to route, easy to document separately.

Cons: Violates REST purist principle (URL should identify a resource, not a version), encourages copy-paste versioning where entire APIs are duplicated.

Used by: Twitter, Facebook, Stripe (partially), most public APIs.

Strategy 2: Header Versioning

GET /users HTTP/1.1
API-Version: 2

Pros: Clean URLs, resource URLs remain stable.

Cons: Can't test in browser, harder to cache (must vary by header), less visible.

Used by: GitHub, Microsoft.

// GitHub example
GET /repos/octocat/Hello-World HTTP/1.1
Accept: application/vnd.github.v3+json

Strategy 3: Query Parameter Versioning

GET /users?version=2

Pros: Easy to implement, can be changed per-request.

Cons: Ugly, easy to forget, not cacheable properly.

Verdict: Don't use this for versioning. Only acceptable for minor variations.

Strategy 4: Content Negotiation (Media Type Versioning)

GET /users HTTP/1.1
Accept: application/vnd.myapi.v2+json

This is the "most correct" REST approach — versioning is part of content negotiation, not the URL. But it's rarely used in practice because it's complex.

When to Version

Don't version for:

• Adding new optional fields to responses (backward compatible)
• Adding new endpoints
• Adding new optional request parameters

Do version for:

• Removing fields from responses
• Renaming fields
• Changing field types (string → integer)
• Changing authentication mechanism
• Fundamentally changing resource structure

Laravel Versioning Implementation

// routes/api.php
Route::prefix('v1')->name('api.v1.')->group(function () {
    require base_path('routes/api/v1.php');
});

Route::prefix('v2')->name('api.v2.')->group(function () {
    require base_path('routes/api/v2.php');
});

// routes/api/v1.php
Route::apiResource('users', V1\UserController::class);

// routes/api/v2.php
Route::apiResource('users', V2\UserController::class);

// app/Http/Controllers/Api/V1/UserController.php
namespace App\Http\Controllers\Api\V1;

// app/Http/Controllers/Api/V2/UserController.php
namespace App\Http\Controllers\Api\V2;

Tip: Don't duplicate entire controllers. Use a versioned transformer/resource layer and share the same service/repository layer.

// Shared service
class UserService
{
    public function findUser(int $id): User { ... }
}

// V1 transforms data one way
class V1\UserResource extends JsonResource
{
    // V1 format: { "full_name": "Ali Al-Ghamdi" }
    public function toArray($request): array
    {
        return ['full_name' => $this->name];
    }
}

// V2 transforms data differently
class V2\UserResource extends JsonResource
{
    // V2 format: { "first_name": "Ali", "last_name": "Al-Ghamdi" }
    public function toArray($request): array
    {
        [$first, $last] = explode(' ', $this->name, 2);
        return ['first_name' => $first, 'last_name' => $last ?? ''];
    }
}

Deprecation Strategy

// Sunset header (RFC 8594) — tells clients when version will be removed
HTTP/1.1 200 OK
Sunset: Sat, 01 Jan 2027 00:00:00 GMT
Deprecation: true
Link: <https://docs.example.com/migration/v1-to-v2>; rel="deprecation"

---

MODULE 4: Pagination, Filtering, Sorting & Searching

Pagination

Strategy 1: Offset/Limit (Page-Based)

GET /users?page=3&per_page=20
GET /users?offset=40&limit=20

How it works: OFFSET 40 LIMIT 20 in SQL — skip 40 records, take 20.

Pros: Simple, supports random access (jump to page 10), familiar to users.

Cons:

• Page drift: If a record is inserted/deleted mid-pagination, you skip or see duplicate records
• Performance: High offsets are slow (OFFSET 1000000 scans 1M rows)
• Not suitable for real-time data

-- This gets slower as offset grows
SELECT * FROM users ORDER BY created_at DESC LIMIT 20 OFFSET 100000;
-- MySQL must read 100,020 rows, discard 100,000

Strategy 2: Cursor-Based Pagination (recommended for scale)

GET /users?cursor=eyJpZCI6MTAwfQ&per_page=20

How it works: Instead of "skip N rows", use a pointer (cursor) to the last seen record.

-- With cursor (id > 100), this is O(log n) with an index
SELECT * FROM users WHERE id > 100 ORDER BY id ASC LIMIT 20;

Pros:

• Consistent results even with inserts/deletes
• O(log n) performance at any depth
• Required for infinite scroll UIs

Cons:

• No random access (can't jump to page 10)
• Harder to implement
• Cursor must be opaque (base64 encoded) to hide implementation details

Laravel Cursor Pagination

// Controller
public function index(Request $request): JsonResponse
{
    $users = User::query()
        ->orderBy('id')
        ->cursorPaginate($request->per_page ?? 20);

    return response()->json([
        'data' => UserResource::collection($users),
        'meta' => [
            'per_page'     => $users->perPage(),
            'has_more'     => $users->hasMorePages(),
            'next_cursor'  => $users->nextCursor()?->encode(),
            'prev_cursor'  => $users->previousCursor()?->encode(),
        ],
        'links' => [
            'next' => $users->nextPageUrl(),
            'prev' => $users->previousPageUrl(),
        ],
    ]);
}

{
  "data": [...],
  "meta": {
    "per_page": 20,
    "has_more": true,
    "next_cursor": "eyJpZCI6MTIwLCJfcG9pbnRzVG9OZXh0SXRlbXMiOnRydWV9",
    "prev_cursor": null
  },
  "links": {
    "next": "/users?cursor=eyJpZCI6MTIwfQ&per_page=20",
    "prev": null
  }
}

Strategy 3: Keyset Pagination (for time-series data)

GET /events?after=2026-06-06T10:00:00Z&limit=20
GET /events?before_id=12345&limit=20

Best for feeds, activity logs, anything sorted by time. Used by Twitter/X, Instagram.

---

Filtering

Design Principles

• Filters are query parameters
• Multiple filters are ANDed by default
• Be explicit about operators

Bad Filtering Design

GET /users?q=active    // What does q mean?
GET /users/active      // Wrong — active is not a resource
GET /users?filter=status:active  // Ambiguous operator syntax

Good Filtering Design

GET /users?status=active
GET /users?status=active&role=admin
GET /users?created_after=2026-01-01&created_before=2026-12-31
GET /users?age_min=18&age_max=65
GET /orders?status[]=pending&status[]=processing  // Multi-value filter

Advanced Filter Operators

For complex filtering needs (avoid over-engineering for simple APIs):

GET /products?price[gte]=100&price[lte]=500
GET /products?name[contains]=phone
GET /users?email[endswith]=@gmail.com

Laravel Filtering with Spatie QueryBuilder

use Spatie\QueryBuilder\QueryBuilder;
use Spatie\QueryBuilder\AllowedFilter;

public function index(Request $request): JsonResponse
{
    $users = QueryBuilder::for(User::class)
        ->allowedFilters([
            'name',
            'email',
            AllowedFilter::exact('status'),
            AllowedFilter::exact('role'),
            AllowedFilter::scope('created_after'),
            AllowedFilter::scope('created_before'),
            AllowedFilter::callback('search', function ($query, $value) {
                $query->where(function ($q) use ($value) {
                    $q->where('name', 'like', "%{$value}%")
                      ->orWhere('email', 'like', "%{$value}%");
                });
            }),
        ])
        ->allowedSorts(['name', 'email', 'created_at', 'orders_count'])
        ->allowedIncludes(['orders', 'profile'])
        ->paginate($request->per_page ?? 20);

    return UserResource::collection($users)->response();
}

// Usage
GET /users?filter[status]=active&filter[role]=admin&sort=-created_at&include=orders

---

Sorting

GET /users?sort=name           // Ascending by name
GET /users?sort=-created_at    // Descending (minus prefix)
GET /users?sort=-created_at,name  // Multi-field sort

The - prefix convention (used by JSON:API spec) is elegant and widely adopted.

Never sort by non-indexed columns in production. Always add database indexes for sortable fields.

---

Searching

Full-Text Search vs Filtering

• Filtering: Exact or range matches on structured data (status=active)
• Searching: Relevance-ranked matching on unstructured text

GET /products?search=wireless+headphones
GET /users?q=ali@  // Prefix search on email

Laravel Scout + Meilisearch/Elasticsearch

// For full-text search, use Laravel Scout
class Product extends Model
{
    use Searchable;

    public function toSearchableArray(): array
    {
        return [
            'id'          => $this->id,
            'name'        => $this->name,
            'description' => $this->description,
            'tags'        => $this->tags->pluck('name')->toArray(),
        ];
    }
}

// Controller
public function search(Request $request): JsonResponse
{
    $results = Product::search($request->q)
        ->where('status', 'active')
        ->paginate(20);

    return ProductResource::collection($results)->response();
}

---

Field Selection (Sparse Fieldsets)

Allow clients to request only the fields they need. Critical for mobile APIs where bandwidth matters.

GET /users?fields=id,name,email
GET /users?fields[users]=id,name&fields[orders]=id,total

Response with field selection:

{
  "data": [
    { "id": 1, "name": "Ali" },
    { "id": 2, "name": "Omar" }
  ]
}

This reduces payload size, reduces database query cost, and improves mobile performance.

// Laravel implementation
public function index(Request $request): JsonResponse
{
    $allowedFields = ['id', 'name', 'email', 'phone', 'created_at'];
    $requestedFields = $request->has('fields')
        ? array_intersect(
            explode(',', $request->fields),
            $allowedFields
        )
        : $allowedFields;

    $users = User::select($requestedFields)->paginate(20);

    return response()->json(['data' => $users]);
}

---

MODULE 5: Authentication & Authorization

Authentication

Authentication answers: Who are you?

API Key Authentication

Simplest form. A random token issued per application.

GET /users HTTP/1.1
X-API-Key: sk_live_abc123xyz

Pros: Simple, easy to revoke, no expiry complexity.
Cons: Long-lived credentials, single factor, hard to scope per-request.

Use for: Server-to-server communication, third-party integrations, webhooks.

JWT (JSON Web Tokens)

Self-contained tokens that carry claims. The server doesn't need to look up a database to validate.

Authorization: Bearer eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9...

JWT Structure: header.payload.signature

// Decoded payload
{
  "sub": "user_5",
  "iss": "https://api.example.com",
  "aud": "https://api.example.com",
  "iat": 1717660800,
  "exp": 1717664400,
  "jti": "unique-token-id",
  "role": "admin",
  "scopes": ["users:read", "orders:write"]
}

Critical JWT security rules:

• Use RS256 (asymmetric) not HS256 (symmetric) for multi-service systems
• Always validate exp, iss, aud
• Short expiry on access tokens (15 min - 1 hour)
• Never store sensitive data in JWT payload (it's base64, not encrypted)
• Include jti (JWT ID) for token revocation capability

OAuth 2.0 + PKCE (for user-facing apps)

The industry standard for delegated authorization. When your API is accessed on behalf of users by third-party apps.

Client → Authorization Server → Access Token + Refresh Token
Client → Resource Server (API) with Access Token

Grant Types:

• Authorization Code + PKCE: Web/mobile apps acting on behalf of users
• Client Credentials: Server-to-server (machine-to-machine)
• Refresh Token: Get new access token without re-auth

Laravel Sanctum vs Passport

Feature	Sanctum	Passport
SPA Authentication	✅ (cookie-based)	❌
Mobile API tokens	✅	✅
OAuth 2.0 server	❌	✅
Third-party apps	❌	✅
Complexity	Low	High
Use case	First-party clients	Third-party OAuth

// Laravel Sanctum — token with abilities (scopes)
$token = $user->createToken('mobile-app', ['orders:read', 'orders:write']);

// In controller — check ability
$request->user()->tokenCan('orders:write')
    || abort(403, 'Token cannot write orders');

---

Authorization

Authorization answers: Are you allowed to do this?

RBAC vs ABAC

RBAC (Role-Based Access Control):

User → Role → Permissions

Simple, fast, works for most applications.

// Roles: admin, manager, user
// Permissions: users:read, users:write, orders:delete

// Laravel Gates & Policies
class OrderPolicy
{
    public function update(User $user, Order $order): bool
    {
        // Role check
        if ($user->hasRole('admin')) return true;

        // Ownership check
        return $user->id === $order->user_id;
    }

    public function delete(User $user, Order $order): bool
    {
        return $user->hasRole('admin')
            && $order->status === 'pending';
    }
}

// In controller
$this->authorize('update', $order);

ABAC (Attribute-Based Access Control):
Authorization based on attributes of the user, resource, and environment:

User(role=manager, department=finance) can access Order(department=finance) during BusinessHours

More powerful, more complex. Use for enterprise systems with complex access rules.

Scope-Based Authorization (for API tokens)

// Define scopes
// users:read, users:write, orders:read, orders:write, admin:*

// Check scope in middleware
Route::middleware('auth:sanctum', 'scope:orders:write')
    ->patch('/orders/{order}', [OrderController::class, 'update']);

// Or in controller
if (!$request->user()->tokenCan('orders:write')) {
    abort(403, 'Insufficient scope. Required: orders:write');
}

---

MODULE 6: Rate Limiting

The Concept

Rate limiting protects your API from abuse, ensures fair resource distribution, and prevents accidental or malicious DoS attacks.

Algorithms

1. Fixed Window Counter

Count requests in fixed time windows (e.g., per minute).

Window: 10:00:00 - 10:01:00
Requests: 98/100 used
At 10:01:00 → counter resets to 0

Problem: Burst at window boundary — a client can make 100 requests at 10:00:59 and 100 more at 10:01:01 (200 requests in 2 seconds).

2. Sliding Window Log

Track timestamps of all requests, count within rolling window.

Pros: No boundary burst problem.
Cons: Memory intensive (must store all timestamps).

3. Token Bucket (most widely used)

A bucket holds tokens. Tokens refill at a fixed rate. Each request consumes one token.

Bucket capacity: 100 tokens
Refill rate: 10 tokens/second
Request cost: 1 token

Allows bursting up to 100 requests, then sustained 10 req/s

Used by: AWS, Stripe, most major APIs.

4. Leaky Bucket

Requests queue, process at fixed rate. Smooths bursts entirely.

Rate Limit Response

HTTP/1.1 429 Too Many Requests
X-RateLimit-Limit: 1000
X-RateLimit-Remaining: 0
X-RateLimit-Reset: 1717664400
Retry-After: 3600
Content-Type: application/json

{
  "error": {
    "type": "rate_limit_exceeded",
    "title": "Too Many Requests",
    "status": 429,
    "detail": "You have exceeded 1000 requests per hour.",
    "retry_after": 3600,
    "limit": 1000,
    "reset_at": "2026-06-06T11:00:00Z"
  }
}

Tiered Rate Limiting

Free tier:      100 requests/hour
Pro tier:       10,000 requests/hour
Enterprise:     Unlimited (but with burst protection)

Per-endpoint limits:
POST /payments: 10/minute (strict — expensive operation)
GET /products:  1000/minute (lenient — cheap read)

Laravel Implementation

// RouteServiceProvider or routes/api.php
RateLimiter::for('api', function (Request $request) {
    return $request->user()
        ? Limit::perHour(1000)->by($request->user()->id)
        : Limit::perMinute(30)->by($request->ip());
});

RateLimiter::for('payments', function (Request $request) {
    return [
        Limit::perMinute(10)->by($request->user()->id),
        Limit::perDay(500)->by($request->user()->id),
    ];
});

// Apply in routes
Route::middleware(['auth:sanctum', 'throttle:payments'])
    ->post('/payments', [PaymentController::class, 'store']);

// Custom throttle middleware with proper headers
class ThrottleWithHeaders
{
    public function handle(Request $request, Closure $next, string $limiterName): Response
    {
        $limiter = RateLimiter::limiter($limiterName);
        $key     = $request->user()?->id ?? $request->ip();
        $limit   = 1000;
        $remaining = RateLimiter::remaining($limiterName . ':' . $key, $limit);

        $response = $next($request);

        return $response
            ->header('X-RateLimit-Limit',     $limit)
            ->header('X-RateLimit-Remaining', max(0, $remaining))
            ->header('X-RateLimit-Reset',     RateLimiter::availableIn($limiterName . ':' . $key));
    }
}

---

MODULE 7: Caching

The Concept

Caching is the most impactful performance optimization in API design. A cache hit is orders of magnitude faster than a database query.

HTTP Caching Headers

Cache-Control Directives

// Public cache (CDN can cache this)
Cache-Control: public, max-age=3600

// Private cache (only client can cache, not CDN)
Cache-Control: private, max-age=300

// Never cache (user-specific, sensitive data)
Cache-Control: no-store

// Cache but always revalidate with server
Cache-Control: no-cache, must-revalidate

// Stale-while-revalidate: serve stale while fetching fresh
Cache-Control: max-age=300, stale-while-revalidate=60

ETags (Entity Tags)

Enables conditional requests — client asks "has this changed since I last saw it?"

// First request
GET /users/5
HTTP/1.1 200 OK
ETag: "abc123def456"
Cache-Control: private, max-age=300

// Subsequent request (client sends ETag back)
GET /users/5
If-None-Match: "abc123def456"

HTTP/1.1 304 Not Modified
// No body sent — saves bandwidth

Last-Modified

GET /products/10
HTTP/1.1 200 OK
Last-Modified: Wed, 05 Jun 2026 10:00:00 GMT

GET /products/10
If-Modified-Since: Wed, 05 Jun 2026 10:00:00 GMT
HTTP/1.1 304 Not Modified

Application-Level Caching Strategy

// Cache expensive queries
public function show(int $id): JsonResponse
{
    $user = Cache::tags(['users', "user:{$id}"])
        ->remember("user:{$id}", now()->addMinutes(15), fn() =>
            User::with(['orders', 'profile'])->findOrFail($id)
        );

    return (new UserResource($user))
        ->response()
        ->header('ETag', '"' . md5($user->updated_at) . '"')
        ->header('Cache-Control', 'private, max-age=900');
}

// Invalidate on update
public function update(UpdateUserRequest $request, User $user): JsonResponse
{
    $user->update($request->validated());

    Cache::tags(["user:{$user->id}", 'users'])->flush();

    return new UserResource($user->fresh());
}

Caching Strategies

Strategy	When to Use	Example
Cache-Aside	Read-heavy, tolerate slight staleness	Product catalog
Write-Through	Write and update cache together	User profile
Cache-Around-DB	Full page/response cache	Public homepage API
CDN Caching	Public, static-ish data	Product images, public listings

---

MODULE 8: API Security

OWASP API Top 10 (2023)

1. Broken Object Level Authorization (BOLA/IDOR) — #1 vulnerability
2. Broken Authentication
3. Broken Object Property Level Authorization
4. Unrestricted Resource Consumption
5. Broken Function Level Authorization
6. Unrestricted Access to Sensitive Business Flows
7. Server Side Request Forgery (SSRF)
8. Security Misconfiguration
9. Improper Inventory Management
10. Unsafe Consumption of APIs

BOLA/IDOR — The Deadliest API Vulnerability

// VULNERABLE: Uses user-supplied ID directly
public function show(int $orderId): JsonResponse
{
    $order = Order::findOrFail($orderId);
    // If user requests /orders/999 (someone else's order) → exposed!
    return new OrderResource($order);
}

// SECURE: Always scope to authenticated user
public function show(int $orderId): JsonResponse
{
    $order = Order::where('user_id', auth()->id())
        ->findOrFail($orderId);
    return new OrderResource($order);
}

// EVEN BETTER: Use Policy
public function show(Order $order): JsonResponse
{
    $this->authorize('view', $order);
    return new OrderResource($order);
}

Security Headers

// Middleware: SecurityHeaders.php
class SecurityHeaders
{
    public function handle(Request $request, Closure $next): Response
    {
        $response = $next($request);

        return $response
            ->header('X-Content-Type-Options',    'nosniff')
            ->header('X-Frame-Options',            'DENY')
            ->header('X-XSS-Protection',           '1; mode=block')
            ->header('Strict-Transport-Security',  'max-age=31536000; includeSubDomains')
            ->header('Content-Security-Policy',    "default-src 'none'")
            ->header('Referrer-Policy',            'strict-origin-when-cross-origin')
            ->header('Permissions-Policy',         'geolocation=()')
            ->header('Cache-Control',              'no-store') // for sensitive endpoints
            ->removeHeader('X-Powered-By')   // Remove server fingerprinting
            ->removeHeader('Server');
    }
}

Input Sanitization

// Never trust client input — parameterized queries always
// Laravel's ORM handles this, but raw queries need care:

// VULNERABLE
DB::select("SELECT * FROM users WHERE email = '{$request->email}'");

// SECURE
DB::select("SELECT * FROM users WHERE email = ?", [$request->email]);
// Or via Eloquent (always parameterized)
User::where('email', $request->email)->first();

---

MODULE 9: Webhooks

The Concept

Webhooks are reverse APIs — instead of your client polling your API ("are there new events?"), your API pushes events to the client when they happen.

Traditional API (Pull):
Client → "Any new orders?" → Server
Client ← "No" ← Server
[repeat every 60 seconds]

Webhook (Push):
Server → "New order created" → Client's endpoint

Webhook Design

POST https://customer.example.com/webhooks/my-api HTTP/1.1
Content-Type: application/json
X-Webhook-ID: evt_abc123
X-Webhook-Timestamp: 1717660800
X-Webhook-Signature: sha256=computed-hmac-signature

{
  "id": "evt_abc123",
  "type": "order.created",
  "created_at": "2026-06-06T10:00:00Z",
  "api_version": "2026-06-01",
  "data": {
    "object": {
      "id": "order_xyz",
      "status": "pending",
      "total": 15000,
      "currency": "SAR"
    }
  }
}

Webhook Signature Verification

Compute HMAC-SHA256 of the payload using a shared secret. The receiver verifies the signature before processing.

// Sending a webhook (in your API)
class WebhookDispatcher
{
    public function dispatch(string $endpoint, array $payload, string $secret): void
    {
        $body      = json_encode($payload);
        $timestamp = time();
        $signature = hash_hmac('sha256', "{$timestamp}.{$body}", $secret);

        Http::withHeaders([
            'Content-Type'           => 'application/json',
            'X-Webhook-Timestamp'    => $timestamp,
            'X-Webhook-Signature'    => "sha256={$signature}",
            'X-Webhook-ID'           => $payload['id'],
        ])->post($endpoint, $payload);
    }
}

// Receiving a webhook (in your client)
class WebhookController extends Controller
{
    public function handle(Request $request): JsonResponse
    {
        $this->verifySignature($request);
        $this->handleIdempotency($request->header('X-Webhook-ID'));

        match($request->input('type')) {
            'order.created'  => $this->handleOrderCreated($request->input('data')),
            'order.paid'     => $this->handleOrderPaid($request->input('data')),
            'payment.failed' => $this->handlePaymentFailed($request->input('data')),
            default          => null, // Unknown event type — silently ignore
        };

        return response()->json(['received' => true]);
    }

    private function verifySignature(Request $request): void
    {
        $secret    = config('webhooks.secret');
        $timestamp = $request->header('X-Webhook-Timestamp');
        $provided  = $request->header('X-Webhook-Signature');
        $body      = $request->getContent();

        // Prevent replay attacks — reject events older than 5 minutes
        if (abs(time() - $timestamp) > 300) {
            abort(401, 'Webhook timestamp too old');
        }

        $expected = 'sha256=' . hash_hmac('sha256', "{$timestamp}.{$body}", $secret);

        if (!hash_equals($expected, $provided)) {
            abort(401, 'Invalid webhook signature');
        }
    }
}

Webhook Reliability Patterns

1. Retry with exponential backoff: If delivery fails, retry after 1m, 5m, 30m, 2h, 24h
2. Idempotency: Consumers must handle duplicate delivery (use webhook ID)
3. Dead letter queue: After N retries, move to DLQ for manual inspection
4. Event ordering: Don't rely on order — include timestamps and let consumers sort
5. Event schema versioning: Include api_version in payload

---

MODULE 10: Pagination Deep Dive — Stripe Pattern

Stripe uses cursor-based pagination with a unique design:

GET /v1/charges?limit=3&starting_after=ch_abc123

{
  "object": "list",
  "data": [
    { "id": "ch_def456", ... },
    { "id": "ch_ghi789", ... },
    { "id": "ch_jkl012", ... }
  ],
  "has_more": true,
  "url": "/v1/charges",
  "next_page": "/v1/charges?limit=3&starting_after=ch_jkl012"
}

The cursor (starting_after) is the actual resource ID — no encoding needed. Simple and transparent.

---

MODULE 11: HATEOAS

The Concept

HATEOAS (Hypermedia As The Engine Of Application State) is Roy Fielding's Level 3 REST maturity. The API tells clients what they can do next via links in the response.

{
  "id": "order_123",
  "status": "pending",
  "total": 15000,
  "_links": {
    "self":    { "href": "/orders/123" },
    "cancel":  { "href": "/orders/123/cancel", "method": "POST" },
    "payment": { "href": "/orders/123/payment", "method": "POST" },
    "customer":{ "href": "/users/5" }
  },
  "_actions": {
    "can_cancel":  true,
    "can_pay":     true,
    "can_ship":    false
  }
}

The client doesn't hardcode URLs — it discovers them from the response. The server can change URLs without breaking clients (they follow links, not hardcoded paths).

Reality check: Pure HATEOAS is rare in practice. Most APIs include a simplified version — just enough links to guide navigation without the full hypermedia engine.

---

MODULE 12: API Versioning Evolution Strategies

Backward Compatibility Rules

Always safe:

• Adding new endpoints
• Adding new optional fields to responses
• Adding new optional request parameters
• Adding new enum values to extensible lists

Never safe (requires version bump):

• Removing fields from responses
• Renaming fields
• Changing field types
• Making optional parameters required
• Changing authentication mechanisms

The Expand/Contract Pattern

Instead of breaking changes, expand first, then contract:

Phase 1 (Expand): Add new field alongside old field
{
  "full_name": "Ali Al-Ghamdi",     ← keep old field
  "first_name": "Ali",               ← add new fields
  "last_name": "Al-Ghamdi"
}

Phase 2: Announce deprecation of full_name
Deprecation: true (header)

Phase 3 (Contract): Remove full_name in v2
{
  "first_name": "Ali",
  "last_name": "Al-Ghamdi"
}

---

MODULE 13: GraphQL vs REST

When REST Wins

• Simple, well-defined resource relationships
• Public APIs that need to be simple to consume
• Strong caching requirements (GraphQL queries are hard to cache)
• When you need HTTP-level tooling (CDNs, reverse proxies, rate limiting by endpoint)
• File uploads (GraphQL is awkward here)

When GraphQL Wins

• Complex, highly-interconnected data (social graphs, GitHub's repo/PR/issue relationships)
• Multiple clients with different data needs (mobile vs web vs third-party)
• Rapid frontend development (frontend teams can get exactly what they need)
• Avoiding over-fetching and under-fetching

The N+1 Problem in GraphQL

query {
  users {        # 1 query
    name
    orders {     # N queries (one per user) ← N+1 problem
      total
    }
  }
}

Solved with DataLoader (batching). REST avoids this with ?include=orders and eager loading.

GitHub's Dual Approach

GitHub ran REST v3 for years, then built GraphQL v4 for complex queries. They kept both — REST for simplicity, GraphQL for power. This is a mature, pragmatic approach.

---

MODULE 14: Bulk Operations

The Problem

Creating 1000 records one by one = 1000 HTTP requests. That's catastrophic for both client and server.

Design Patterns

Batch Create

POST /users/bulk HTTP/1.1
Content-Type: application/json

{
  "users": [
    { "name": "User One", "email": "[e-mail]" },
    { "name": "User Two", "email": "[e-mail]" }
  ]
}

HTTP/1.1 207 Multi-Status
Content-Type: application/json

{
  "results": [
    { "index": 0, "status": 201, "id": 5, "data": { ... } },
    { "index": 1, "status": 422, "error": { "email": ["already taken"] } }
  ],
  "summary": {
    "total": 2,
    "succeeded": 1,
    "failed": 1
  }
}

207 Multi-Status is the correct code when a batch has partial success.

Async Bulk Operations

For large batches, process asynchronously:

POST /imports HTTP/1.1
Content-Type: multipart/form-data

{ file: users.csv }

HTTP/1.1 202 Accepted
Location: /imports/import_abc123

{
  "id": "import_abc123",
  "status": "processing",
  "progress_url": "/imports/import_abc123",
  "estimated_completion": "2026-06-06T10:05:00Z"
}

GET /imports/import_abc123

{
  "id": "import_abc123",
  "status": "completed",
  "total": 10000,
  "processed": 10000,
  "succeeded": 9987,
  "failed": 13,
  "errors_url": "/imports/import_abc123/errors"
}

---

MODULE 15: API Observability & Monitoring

The Three Pillars

1. Metrics

• Request rate (requests per second per endpoint)
• Error rate (4xx, 5xx percentage)
• Latency (p50, p95, p99 — not just average)
• Upstream dependency health

// Laravel — emit metrics per request
class ApiMetricsMiddleware
{
    public function handle(Request $request, Closure $next): Response
    {
        $start = microtime(true);

        $response = $next($request);

        $duration = (microtime(true) - $start) * 1000; // ms

        // Emit to your metrics system (DataDog, Prometheus, etc.)
        app('metrics')->histogram('api.request.duration', $duration, [
            'endpoint'  => $request->route()?->getName(),
            'method'    => $request->method(),
            'status'    => $response->getStatusCode(),
            'version'   => $request->segment(1), // v1, v2
        ]);

        return $response->header('X-Response-Time', round($duration, 2) . 'ms');
    }
}

2. Logging — Structured JSON Logs

// Structured log entry for every API request
Log::channel('api')->info('api_request', [
    'trace_id'    => $request->header('X-Request-ID'),
    'method'      => $request->method(),
    'path'        => $request->path(),
    'status'      => $response->getStatusCode(),
    'duration_ms' => $duration,
    'user_id'     => auth()->id(),
    'ip'          => $request->ip(),
    'user_agent'  => $request->userAgent(),
]);

3. Tracing — Distributed Tracing

Every request gets a trace_id. This ID is propagated across all services so you can see the entire flow of a request through your microservices.

// RequestIdMiddleware.php
class RequestIdMiddleware
{
    public function handle(Request $request, Closure $next): Response
    {
        $traceId = $request->header('X-Request-ID') ?? Str::uuid()->toString();

        // Make available throughout the request
        $request->headers->set('X-Request-ID', $traceId);
        app()->instance('trace_id', $traceId);

        // Pass to downstream services
        $response = $next($request);

        return $response->header('X-Request-ID', $traceId);
    }
}

---

MODULE 16: OpenAPI / Swagger Documentation

API-First Design

The most mature approach: write the OpenAPI spec before writing any code. This forces you to think about the API design as a contract, not as an afterthought.

openapi: 3.1.0
info:
  title: My API
  version: 2.0.0
  description: |
    Complete API for the platform.

    ## Authentication
    Use Bearer token in Authorization header.

    ## Rate Limiting
    1000 requests/hour per API key.

servers:
  - url: https://api.example.com/v2
    description: Production

paths:
  /users:
    get:
      summary: List users
      operationId: listUsers
      tags: [Users]
      security:
        - BearerAuth: []
      parameters:
        - name: page
          in: query
          schema: { type: integer, default: 1, minimum: 1 }
        - name: per_page
          in: query
          schema: { type: integer, default: 20, maximum: 100 }
        - name: status
          in: query
          schema:
            type: string
            enum: [active, inactive, suspended]
        - name: sort
          in: query
          schema:
            type: string
            example: "-created_at"
      responses:
        '200':
          description: Paginated list of users
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/UserCollection'
        '401':
          $ref: '#/components/responses/Unauthorized'
        '429':
          $ref: '#/components/responses/RateLimitExceeded'

components:
  schemas:
    User:
      type: object
      properties:
        id:
          type: integer
          readOnly: true
          example: 5
        name:
          type: string
          example: "Ali Al-Ghamdi"
        email:
          type: string
          format: email
        status:
          type: string
          enum: [active, inactive, suspended]
        created_at:
          type: string
          format: date-time
          readOnly: true

  securitySchemes:
    BearerAuth:
      type: http
      scheme: bearer
      bearerFormat: JWT

Laravel with Scribe or L5-Swagger

// Using Scribe (recommended for Laravel)
composer require knuckleswtf/scribe --dev

// Annotate your controllers
/**
 * @group Users
 *
 * APIs for managing users.
 */
class UserController extends Controller
{
    /**
     * List Users
     *
     * Get a paginated list of users.
     *
     * @queryParam page int Page number. Example: 1
     * @queryParam per_page int Items per page (max 100). Example: 20
     * @queryParam status string Filter by status. Example: active
     *
     * @response 200 {
     *   "data": [{"id": 1, "name": "Ali"}],
     *   "meta": {"total": 100, "per_page": 20}
     * }
     */
    public function index() { ... }
}

---

MODULE 17: File Uploads

Design Considerations

Small Files (< 5MB): Direct Upload

POST /users/5/avatar HTTP/1.1
Content-Type: multipart/form-data; boundary=boundary

--boundary
Content-Disposition: form-data; name="avatar"; filename="photo.jpg"
Content-Type: image/jpeg

[binary data]
--boundary--

HTTP/1.1 200 OK
{
  "avatar_url": "https://cdn.example.com/avatars/5/photo.jpg"
}

Large Files: Pre-signed URL Upload

// Step 1: Request an upload URL
POST /uploads/presigned-url HTTP/1.1
{
  "filename": "large-video.mp4",
  "content_type": "video/mp4",
  "size": 524288000
}

HTTP/1.1 200 OK
{
  "upload_id": "upload_abc123",
  "upload_url": "https://s3.amazonaws.com/bucket/key?X-Amz-Signature=...",
  "expires_at": "2026-06-06T10:15:00Z"
}

// Step 2: Upload directly to S3 (client → S3, never touches your server)
PUT https://s3.amazonaws.com/bucket/key?X-Amz-Signature=...
Content-Type: video/mp4
[file binary]

// Step 3: Confirm upload
POST /uploads/upload_abc123/confirm HTTP/1.1
HTTP/1.1 200 OK
{ "file_url": "https://cdn.example.com/files/abc123/video.mp4" }

This pattern offloads bandwidth from your servers to S3 directly. Your API only handles metadata.

// Laravel S3 pre-signed URL
public function generatePresignedUrl(Request $request): JsonResponse
{
    $request->validate([
        'filename'     => ['required', 'string'],
        'content_type' => ['required', Rule::in(['image/jpeg', 'image/png', 'video/mp4'])],
        'size'         => ['required', 'integer', 'max:' . 500 * 1024 * 1024], // 500MB max
    ]);

    $key      = 'uploads/' . auth()->id() . '/' . Str::uuid() . '/' . $request->filename;
    $s3Client = Storage::disk('s3')->getClient();

    $command = $s3Client->getCommand('PutObject', [
        'Bucket'       => config('filesystems.disks.s3.bucket'),
        'Key'          => $key,
        'ContentType'  => $request->content_type,
    ]);

    $presignedUrl = $s3Client->createPresignedRequest($command, '+15 minutes');

    $upload = Upload::create([
        'user_id'      => auth()->id(),
        's3_key'       => $key,
        'filename'     => $request->filename,
        'content_type' => $request->content_type,
        'size'         => $request->size,
        'status'       => 'pending',
    ]);

    return response()->json([
        'upload_id'  => $upload->id,
        'upload_url' => (string) $presignedUrl->getUri(),
        'expires_at' => now()->addMinutes(15)->toISOString(),
    ]);
}

---

MODULE 18: Microservice API Design

API Gateway Pattern

Client → API Gateway → [Auth Service]
                     → [User Service]
                     → [Order Service]
                     → [Payment Service]

The gateway handles:

• Authentication (verify JWT once, pass claims downstream)
• Rate limiting
• Request routing
• SSL termination
• Request/response logging
• API versioning routing

Service-to-Service Communication

// Internal service calls — use service tokens, not user tokens
// Service A calling Service B
class OrderServiceClient
{
    public function getOrder(string $orderId): array
    {
        return Http::withHeaders([
            'Authorization' => 'Bearer ' . config('services.order_service.secret'),
            'X-Service-Name' => 'payment-service',
            'X-Request-ID'  => app('trace_id'), // Propagate trace ID
        ])
        ->timeout(5)
        ->retry(3, 100, fn($e) => $e instanceof ConnectionException)
        ->get(config('services.order_service.url') . "/internal/orders/{$orderId}")
        ->throw()
        ->json();
    }
}

Internal vs Public API Design

Concern	Public API	Internal API
Versioning	Strict (v1, v2)	Relaxed (but still versioned)
Auth	OAuth/JWT	Service tokens, mTLS
Rate limiting	Yes, by API key	Yes, by service
Documentation	Extensive, public	Internal wiki
Stability	Extremely high	Moderate
Payload size	Minimal (mobile concern)	Richer data OK

---

MODULE 19: Event-Driven APIs

REST vs Event-Driven

REST is synchronous request/response. Event-driven is asynchronous — you emit events, subscribers react when ready.

Event Schema Design

{
  "event_id": "evt_abc123",
  "event_type": "order.status_changed",
  "event_version": "1.0",
  "occurred_at": "2026-06-06T10:00:00Z",
  "producer": "order-service",
  "aggregate_id": "order_xyz",
  "aggregate_type": "Order",
  "correlation_id": "req_abc123",
  "causation_id": "evt_prev456",
  "payload": {
    "order_id": "order_xyz",
    "previous_status": "pending",
    "new_status": "processing",
    "changed_by": "user_5"
  }
}

Event Sourcing with Laravel

// Every state change is an event, stored in an event log
class OrderStatusChanged
{
    public function __construct(
        public readonly string $orderId,
        public readonly string $previousStatus,
        public readonly string $newStatus,
        public readonly int    $changedByUserId,
        public readonly Carbon $occurredAt = new Carbon(),
    ) {}
}

class Order extends AggregateRoot
{
    private string $status;

    public function changeStatus(string $newStatus, int $userId): self
    {
        if (!$this->canTransitionTo($newStatus)) {
            throw new InvalidStatusTransition($this->status, $newStatus);
        }

        $this->recordThat(new OrderStatusChanged(
            orderId:        $this->uuid,
            previousStatus: $this->status,
            newStatus:      $newStatus,
            changedByUserId: $userId,
        ));

        return $this;
    }

    protected function applyOrderStatusChanged(OrderStatusChanged $event): void
    {
        $this->status = $event->newStatus;
    }
}

---

PRACTICAL EXERCISES, INTERVIEW QUESTIONS & REAL-WORLD SCENARIOS

---

Module 1: Resource Design

Exercise: Design a RESTful API for a library system. You have books, authors, members, loans, and reservations. Draw out all the resources, their relationships, and every endpoint you'd expose.

Interview Question: "We have an endpoint POST /send-email. Why might this be poorly designed, and how would you fix it?"

Answer to internalize: Actions that aren't CRUD need to be reframed as resource operations. POST /email-campaigns creates a campaign. POST /email-campaigns/{id}/send treats "send" as a state change on a resource. The key is that state transitions become POST/PATCH operations on resources.

Real-World Scenario: Your team built GET /getActiveUsersByDepartment?dept=engineering six months ago. 50 third-party integrations use it. Design a migration strategy to move to proper REST design without breaking any consumers.

---

Module 2: Idempotency

Exercise: Implement a payment API with idempotency key support. Handle the case where the payment processor returns a network timeout — the idempotency layer should prevent double charges even across server restarts.

Interview Question: "Explain how you'd implement an idempotent API for transferring money between bank accounts. What happens if the database write succeeds but the response never reaches the client?"

Real-World Scenario: Your payment service processed 10,000 transactions yesterday. This morning you find 47 duplicate charges — the mobile app was retrying failed requests without idempotency keys. Design a post-incident fix and a preventive architecture.

---

Module 3: Versioning

Exercise: You have a GET /users/{id} endpoint that returns { "name": "Full Name" }. Product wants to split this into first_name and last_name. Design a versioning strategy that doesn't break existing clients.

Interview Question: "Stripe has maintained API backward compatibility since 2011. How is this architecturally possible? What are the trade-offs?"

Real-World Scenario: Your v1 API has 200 enterprise customers. You need to change the authentication system from API keys to OAuth 2.0 in v2. Design the migration plan, including what happens to v1 customers.

---

Module 4: Security

Exercise: Audit this endpoint for security vulnerabilities and fix all of them:

public function getUserOrders(Request $request): JsonResponse
{
    $userId = $request->input('user_id');
    $orders = DB::select("SELECT * FROM orders WHERE user_id = $userId");
    return response()->json($orders);
}

Interview Question: "What is BOLA/IDOR and how would you design a system to prevent it systematically, rather than relying on individual developers to check ownership in every controller?"

Real-World Scenario: You're designing a multi-tenant SaaS API where Company A must never see Company B's data. Design the authorization architecture. What happens if a developer forgets to add the tenant scope to a query?

---

Module 5: System Design Integration

Exercise: Design a complete API for a FinTech payment platform similar to HyperPay or Stripe. Include:

• Authentication model
• Idempotency for payments
• Webhook delivery with retry
• Rate limiting by tier
• Audit logging
• Multi-currency support
• Versioning strategy

Interview Question: "Design an API rate limiting system that works across 50 horizontally-scaled API servers. A single Redis node can become a bottleneck — how would you handle this at scale?"

Key insight: Use a distributed token bucket with Redis Cluster. For extreme scale, use approximate counting with a local cache that syncs to Redis every N milliseconds — trading perfect accuracy for throughput.

Real-World Scenario: Your ZATCA e-invoicing API (which you have real experience with) needs to be exposed to 500 Saudi businesses. Design the authentication model, rate limiting tiers, webhook notifications for invoice status, and how you'd handle ZATCA's own API being unavailable (circuit breaker pattern).

---

FINAL ARCHITECTURE SUMMARY

                    ┌─────────────────────────────────────┐
                    │            API Gateway               │
                    │  • Auth Token Validation             │
                    │  • Rate Limiting (by tier)           │
                    │  • Request Logging & Tracing         │
                    │  • SSL Termination                   │
                    │  • Version Routing (v1 → v2)         │
                    └──────────────┬──────────────────────┘
                                   │
              ┌────────────────────┼────────────────────┐
              ▼                    ▼                    ▼
    ┌─────────────────┐  ┌─────────────────┐  ┌─────────────────┐
    │  User Service   │  │  Order Service  │  │ Payment Service │
    │                 │  │                 │  │                 │
    │ • RBAC/ABAC     │  │ • State Machine │  │ • Idempotency   │
    │ • Resource      │  │ • Event Emit    │  │ • Double-entry  │
    │   Policies      │  │ • Cursor Paging │  │ • Webhook Emit  │
    └────────┬────────┘  └────────┬────────┘  └────────┬────────┘
             │                    │                    │
             └────────────────────▼────────────────────┘
                                  │
                    ┌─────────────▼────────────┐
                    │      Event Bus (Kafka)    │
                    │  • order.created         │
                    │  • payment.completed     │
                    │  • user.updated          │
                    └─────────────┬────────────┘
                                  │
                    ┌─────────────▼────────────┐
                    │    Webhook Dispatcher    │
                    │  • Retry with backoff    │
                    │  • Signature HMAC-SHA256 │
                    │  • Dead Letter Queue     │
                    └──────────────────────────┘