DEV Community

Divya Darshana
Divya Darshana

Posted on

Building a Server-Rendered Dev.to scraper : HTML Templating, API Integration, and Pagination

#go #webdev #productivity #softwaredevelopment

Watch Tutorial Video

Click to watch the companion tutorial video

Why Server-Side Rendering (SSR) Matters

In today's web development landscape dominated by client-side frameworks, server-side rendering offers compelling advantages:

  1. Instant Page Loads: HTML arrives ready-to-display
  2. SEO Superiority: Search engines easily crawl content
  3. Simplified Architecture: Single codebase manages both API and UI
  4. Resource Efficiency: Reduced client-side JavaScript overhead

GoFr, an opinionated Go framework, makes SSR implementation remarkably straightforward. Let's build a Dev.to article reader that demonstrates these principles in action.

Project Setup: Laying the Foundation

1. Initializing the GoFr Application (main.go) 

package main

import (
    "encoding/json"
    "gofr.dev/pkg/gofr"
    "strconv"
)

type Article struct {
    Title       string `json:"title"`
    URL         string `json:"url"`
    Description string `json:"description"`
}

type PageData struct {
    Articles []Article
    Tag      string
    Page     int
}

func main() {
    // Initialize GoFr application
    app := gofr.New()

    // Configure Dev.to API service
    app.AddHTTPService("dev-articles", "https://dev.to/api")

    // Register route handler
    app.GET("/dev-articles", FetchArticles)

    // Serve static files (CSS, templates)
    app.AddStaticFiles("/", "./static")

    // Start the server
    app.Run()
}
Enter fullscreen mode Exit fullscreen mode

Key Components Explained:

  • Service Configuration:AddHTTPService creates a pre-configured HTTP client for the Dev.to API, handling connection pooling and timeouts automatically.

  • Route Handling:The GET method associates the /dev-articles endpoint with our FetchArticles handler.

  • Static Assets:AddStaticFiles serves CSS and templates from the static directory, essential for our SSR approach.

Core Business Logic: The Article Handler

2. Implementing the FetchArticles Handler 

func FetchArticles(ctx *gofr.Context) (any, error) {
    // Get search parameters with defaults
    tag := ctx.Param("tag")
    if tag == "" {
        tag = "go" // Default to Go articles
    }

    page, _ := strconv.Atoi(ctx.Param("page"))
    if page < 1 {
        page = 1 // Ensure minimum page number
    }

    // Fetch articles from Dev.to API
    service := ctx.GetHTTPService("dev-articles")
    resp, err := service.Get(ctx, "/articles", map[string]interface{}{
        "tag":      tag,
        "page":     page,
        "per_page": 4, // Optimal for initial load
    })

    if err != nil {
        return nil, err // Handle API errors
    }
    defer resp.Body.Close()

    // Parse API response
    var articles []Article
    if err := json.NewDecoder(resp.Body).Decode(&articles); err != nil {
        return nil, err // Handle parsing errors
    }

    // Render template with data
    return gofr.Template{
        Data: PageData{articles, tag, page},
        Name: "devTo.html",
    }, nil
}
Enter fullscreen mode Exit fullscreen mode

Architectural Decisions:

  1. Parameter Handling

    • Default values ensure consistent behavior
    • Type conversion guards against invalid inputs
    • per_page=4 balances content density and performance

  2. Error Handling

    • Automatic error propagation through GoFr's middleware
    • Clean separation of concerns between API and rendering

  3. Service Abstraction

    • HTTP service configuration centralized in main.go
    • Easy to swap API endpoints or add caching later

Presentation Layer: HTML Templating

3. Template Implementation (static/devTo.html) 

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <title>Go Articles from Dev.to</title>
    <link rel="stylesheet" href="/style.css">
    <script>
        // Hybrid pagination: Client-side navigation with server-side validation
        function updatePage(delta) {
            const params = new URLSearchParams(window.location.search)
            let page = parseInt(params.get('page') || {{.Page}}
            let tag = params.get('tag') || '{{.Tag}}'

            page = Math.max(1, page + delta)
            window.location.href = `/dev-articles?tag=${tag}&page=${page}`
        }
    </script>
</head>

<body>
    <div id="content">
        <h1>📰 Latest Go Articles on Dev.to</h1>

        <!-- Search Form -->
        <form method="GET" action="/dev-articles">
            <input type="text" name="tag" 
                   placeholder="Tag (e.g. go)" 
                   value="{{.Tag}}" />
            <button type="submit">Search</button>
        </form>

        <!-- Article Listing -->
        {{range .Articles}}
        <article class="post">
            <h2><a href="{{.URL}}" target="_blank">{{.Title}}</a></h2>
            <p class="description">{{.Description}}</p>
        </article>
        {{end}}

        <!-- Pagination Controls -->
        <div class="pagination">
            {{if gt .Page 1}}
            <button onclick="updatePage(-1)">⬅️ Prev</button>
            {{else}}
            <button disabled>⬅️ Prev</button>
            {{end}}

            <span class="page-number">Page {{.Page}}</span>

            <button onclick="updatePage(1)">Next ➡️</button>
        </div>
    </div>
</body>
</html>
Enter fullscreen mode Exit fullscreen mode

Template Features:

  1. Dynamic Content Binding

    • {{.Articles}} loop renders server-fetched content
    • {{.Tag}} and {{.Page}} maintain state between requests

  2. Progressive Enhancement

    • Search form works without JavaScript
    • Pagination uses minimal client-side scripting

  3. Security Best Practices

    • Auto-escaping prevents XSS vulnerabilities
    • target="_blank" with rel="noopener" (implicit in Go templates)

Styling: CSS Implementation (static/style.css) 

#content {
    max-width: 800px;
    margin: 0 auto;
    padding: 2rem;
    font-family: system-ui, sans-serif;
}

article.post {
    background: #fff;
    border-radius: 8px;
    padding: 1.5rem;
    margin: 1rem 0;
    box-shadow: 0 2px 4px rgba(0,0,0,0.1);
}

.pagination {
    margin-top: 2rem;
    display: flex;
    gap: 1rem;
    justify-content: center;
    align-items: center;
}

button {
    padding: 0.5rem 1rem;
    background: #3182ce;
    color: white;
    border: none;
    border-radius: 4px;
    cursor: pointer;
}

button:disabled {
    background: #a0aec0;
    cursor: not-allowed;
}

.page-number {
    font-weight: 500;
    color: #4a5568;
}
Enter fullscreen mode Exit fullscreen mode

Design Philosophy:

  1. Modern Aesthetic

    • System font stack for native feel
    • Subtle shadows and rounded corners

  2. Responsive Layout

    • Max-width constraint for readability
    • Flexible spacing using rem units

  3. Accessibility Focus

    • High contrast colors
    • Clear button states (hover/disabled)

Running the Application

  1. Install Dependencies 
   go mod init your-module-name
   go get gofr.dev/pkg/gofr
Enter fullscreen mode Exit fullscreen mode
  1. Directory Structure 
├── templates/
│   └── devTo.html      # HTML templates
├── static/
│   └── style.css       # Static assets
└── main.go             # Application entrypoint
Enter fullscreen mode Exit fullscreen mode
  1. Start the Server 
   go run main.go
Enter fullscreen mode Exit fullscreen mode
  1. Access in Browser Visit http://localhost:8000/dev-articles?tag=go&page=1

Why GoFr Shines in Production

  1. Built-in Observability

    • Automatic request logging
    • Metrics endpoint at /metrics

  2. Configuration Management

   # configs/config.yml
   services:
     dev-articles:
       url: https://dev.to/api
       timeout: 3s
Enter fullscreen mode Exit fullscreen mode

  1. Horizontal Scaling

    • Stateless architecture
    • Native Kubernetes support

  2. Error Handling

    • Centralized error recovery
    • Structured logging

Next Steps: From Prototype to Production

  1. Add Caching Layer 
   // Redis integration example
   cached, err := ctx.Redis.Get(ctx, "articles:"+tag+":"+page)
Enter fullscreen mode Exit fullscreen mode
  1. Implement Rate Limiting 
   app.UseMiddleware(ratelimit.New(100)) // 100 requests/minute
Enter fullscreen mode Exit fullscreen mode
  1. Add Health Checks 
   app.GET("/health", func(ctx *gofr.Context) (any, error) {
       return "OK", nil
   })
Enter fullscreen mode Exit fullscreen mode
  1. Error Boundaries 
   {{if .Error}}
   <div class="error-alert">
       ⚠️ Failed to load articles: {{.Error}}
   </div>
   {{end}}
Enter fullscreen mode Exit fullscreen mode

Conclusion: The Power of Simplicity

This project demonstrates how Go and GoFr enable building modern web applications with:

✅ Full server-side rendering

✅ Clean architecture

✅ Production-ready features

✅ Minimal dependencies

GitHub Repository:

https://github.com/coolwednesday/gofr-template-rendering-example

Ready to Go Further?

  • Explore GoFr's documentation for database integration
  • Implement user authentication using JWT middleware
  • Add server-side caching for API responses

The simplicity of Go combined with GoFr's powerful abstractions makes this stack ideal for projects ranging from small internal tools to large-scale content platforms.

Happy coding! 🚀

Top comments (0)