🚀 Building a Serverless Platform: Systems Programming 🚀

UnplugCharger — Wed, 21 May 2025 09:27:23 +0000

If you've ever felt stuck building yet another todo app and craved a project that dives into the guts of how systems work, you're in the right place. Let me take you through my journey of building a lightweight serverless platform from scratch—a project that taught me more about containerization, process management, and API design than any tutorial ever could.

Heads up: This is an educational project, not production-ready! Check out the full code here.

Why Build a Serverless Platform?

Serverless computing is everywhere, but how does it actually work under the hood? I wanted to peel back the layers and explore:

🛠️ Systems Programming Challenges: File handling, process isolation, security.

🧩 Practical Utility: Build something usable, not just theoretical.

📦 Modern Tech Stack: Docker, REST APIs, multi-language support.

Spoiler: It was harder—and more rewarding—than I expected.

How It Works: Breaking Down the Core Components

1. Handling User Code Safely

When users upload a zip file, the platform must extract it without risking security flaws like the infamous zip slip vulnerability:

// Simplified code snippet for secure zip extraction
filePath := filepath.Join(destDir, file.Name)
if !strings.HasPrefix(filePath, filepath.Clean(destDir)+string(os.PathSeparator)) {
    return fmt.Errorf("illegal file path: %s", file.Name)
}

Key Takeaways:

Validate every file path to prevent directory escapes.
Use Go's os and filepath libraries for safe file operations.

2. Detecting Programming Languages

The platform supports Python and Go (for now!). A simple but effective approach:

// Check for Python files
pythonFiles, _ := filepath.Glob(filepath.Join(dir, "*.py"))
if len(pythonFiles) > 0 {
    return "python", nil
}

Why This Matters:

Introduces pattern matching and filesystem inspection.
Lays groundwork for multi-language support.

3. Docker: The Heart of Isolation

Running untrusted code safely meant leaning into Docker. Here’s the workflow:

Build a Docker image from the user’s code.

Run containers with strict resource limits (CPU, memory, network).

// Simplified Docker execution with security constraints
cmd := exec.Command("docker", "run", 
    "--read-only",     // No writes to filesystem
    "--network=none",  // No internet access
    "--memory=128m",   // Memory cap
    imageTag,
)

Lessons Learned:

Process management in Go using exec.Command.

Security through isolation: Containers run without privileges or network access.

4. The API Layer

A REST API ties everything together. For example, submitting a function:

func submitFunctionHandler(w http.ResponseWriter, r *http.Request) {
    // Handle file upload, extract, build, and store metadata
    respondWithJSON(w, http.StatusCreated, map[string]interface{}{
        "message": fmt.Sprintf("Function '%s' deployed!", function.Name),
    })
}

Design Choices:

Used Go standard library net package for routing.
Middleware for logging, timeout handling, and error recovery.

The Hardest Parts (And What I Learned)

🔒 Security Challenges

Zip Slip Vulnerability: A single path validation oversight could let attackers overwrite system files
Resource Limits: Implementing strict CPU/memory constraints via Docker to prevent abuse
Input Sanitization: Learned to never trust user input - always validate and sanitize

🐛 Debugging Nightmares

Exit Code 124: Spent hours diagnosing Docker's mysterious timeout error
Orphaned Containers: The costly lesson of forgetting --rm flag (thousands of dead containers later...)

What's Missing? (Future Improvements)

While functional, this isn't production-grade like AWS Lambda. Here's the roadmap:

Persistent Storage: Track function metadata and execution history
Authentication Layer: Implement OAuth or API key security
Auto-Scaling: Dynamic container provisioning based on workload
Extended Language Support: Add JavaScript, Rust, and more!

Why You Should Try a Project Like This

Building a serverless platform from scratch taught me:

Systems Thinking: How file I/O, containers, and APIs work together
The Hidden Cost of Serverless: Appreciation for the complexity abstracted away
Production-Grade Resilience: Why robust error handling makes or breaks systems

Final Thoughts

This wasn't about building a better Lambda - it was about understanding the machinery behind the magic. If you're tired of surface-level tutorials, I challenge you to:

Pick a complex systems problem
Dive deeper than any tutorial would
Embrace the struggle (that's where real learning happens)

You'll gain:

Skills most tutorials never cover
Deep respect for the tools you use daily
Confidence to tackle any technical challenge

Ready to tinker? ➡️ Explore the code on GitHub

What's the most ambitious project you've built to level up your skills? Share your war stories below! 💬

DEV Community: UnplugCharger