DEV Community: Lucas de Ataides

Why Clean Architecture Struggles in Golang and What Works Better

Lucas de Ataides — Mon, 04 Nov 2024 19:50:56 +0000

Golang has carved out a solid reputation as a fast, efficient language that prioritizes simplicity, which is one of the reasons why it’s so commonly used for backend services, microservices, and infrastructure tooling. However, as more developers from languages like Java and C# transition to Go, questions about implementing Clean Architecture arise. For those used to Clean Architecture’s layer-based approach to structuring applications, it can feel intuitive to apply the same principles to Go. However, as we’ll explore, trying to implement Clean Architecture in Go often backfires. Instead, we'll look at a structure tailored for Go’s strengths that’s more straightforward, flexible, and aligns with Go’s “keep it simple” philosophy.

Why Clean Architecture Feels Out of Place in Go

The goal of Clean Architecture, championed by Uncle Bob (Robert C. Martin), is to create software that’s modular, testable, and easy to extend. This is achieved by enforcing separation of concerns between layers, with core business logic kept isolated from external concerns. While this works well in highly object-oriented languages like Java, it introduces friction in Go. Here’s why:

1. Go’s Minimalism Fights Against Excessive Abstractions

In Go, there’s a strong emphasis on readability, simplicity, and reduced overhead. Clean Architecture introduces layers upon layers of abstractions: interfaces, dependency inversion, complex dependency injection, and service layers for business logic. However, these extra layers tend to add unnecessary complexity when implemented in Go.

Let’s take Kubernetes as an example. Kubernetes is a massive project built in Go, but it doesn’t rely on Clean Architecture principles. Instead, it embraces a flat, function-oriented structure that’s focused around packages and subsystems. You can see this in the Kubernetes GitHub repository, where packages are organized by functionality rather than rigid layers. By grouping code based on functionality, Kubernetes achieves high modularity without complex abstractions.

The Go philosophy prioritizes practicality and speed. The language’s creators have consistently advocated for avoiding over-architecting, favoring straightforward implementations. If an abstraction isn’t absolutely necessary, it doesn’t belong in Go code. Go’s creators even designed the language without inheritance to avoid the pitfalls of over-engineering, encouraging developers to keep their designs clean and clear.

2. Dependency Injection is Limited by Design

Clean Architecture leans heavily on Dependency Injection to decouple different layers and make modules more testable. In languages like Java, DI is a natural part of the ecosystem thanks to frameworks like Spring. These frameworks handle DI automatically, allowing you to wire dependencies together with ease, without cluttering your code.

However, Go lacks a native DI system, and most DI libraries for Go are either overly complex or feel unidiomatic. Go relies on explicit dependency injection via constructor functions or function parameters, keeping dependencies clear and avoiding “magic” hidden in DI containers. Go’s approach makes code more explicit, but it also means that if you introduce too many layers, the dependency management becomes unmanageable and verbose.

In Kubernetes, for example, you don’t see complex DI frameworks or DI containers. Instead, dependencies are injected in a straightforward manner using constructors. This design keeps the code transparent and avoids the pitfalls of DI frameworks. Golang encourages using DI only where it truly makes sense, which is why Kubernetes avoids creating unnecessary interfaces and dependencies just for the sake of following a pattern.

3. Testing Becomes More Complex with Too Many Layers

Another challenge with Clean Architecture in Go is that it can make testing unnecessarily complicated. In Java, for instance, Clean Architecture supports robust unit testing with heavy use of mocks for dependencies. Mocking allows you to isolate each layer and test it independently. However, in Go, creating mocks can be cumbersome, and the Go community generally favors integration testing or testing with real implementations wherever possible.

In production-grade Go projects, such as Kubernetes, testing isn’t handled by isolating each component but by focusing on integration and end-to-end tests that cover real-life scenarios. By reducing the abstraction layers, Go projects like Kubernetes achieve high test coverage while keeping tests close to actual behavior, which results in more confidence when deploying in production.

The Best Architectural Approach for Golang

So if Clean Architecture doesn’t fit well with Go, what does? The answer lies in a simpler, more functional structure that emphasizes packages and focuses on modularity over strict layering. One effective architectural pattern for Go is based on Hexagonal Architecture, often known as Ports and Adapters. This architecture allows for modularity and flexibility without excessive layering.

The Golang Standards Project Layout is a great starting point for creating production-ready projects in Go. This structure provides a foundation for organizing code by purpose and functionality rather than by architectural layer.

Go Project Structure: A Practical Example

You're absolutely right! Structuring Go projects with a package-focused approach, where functionality is broken down by packages rather than a layered folder structure, aligns better with Go’s design principles. Instead of creating top-level directories by layers (e.g., controllers, services, repositories), it’s more idiomatic in Go to create cohesive packages, each encapsulating its own models, services, and repositories. This package-based approach reduces coupling and keeps code modular, which is essential for a production-grade Go application.

Let’s look at a refined, package-centric structure suited for Go:

/myapp
   /cmd                   // Entrypoints for different executables (e.g., main.go)
      /myapp-api
         main.go          // Entrypoint for the main application
   /config                // Configuration files and setup
   /internal              // Private/internal packages (not accessible externally)
      /user               // Package focused on user-related functionality
         models.go        // Data models and structs specific to user functionality
         service.go       // Core business logic for user operations
         repository.go    // Database access methods for user data
      /order              // Package for order-related logic
         models.go        // Data models for orders
         service.go       // Core order-related logic
         repository.go    // Database access for orders
   /pkg                   // Shared, reusable packages across the application
      /auth               // Authorization and authentication package
      /logger             // Custom logging utilities
   /api                   // Package with REST or gRPC handlers
      /v1
         user_handler.go  // Handler for user-related endpoints
         order_handler.go // Handler for order-related endpoints
   /utils                 // General-purpose utility functions and helpers
   go.mod                 // Module file

Key Components in the Package-Based Structure

/cmd

This folder is the conventional location for the application's entry points. Each subfolder here represents a different executable for the app. For example, in microservice architectures, each service can have its own directory here with its main.go. The code here should be minimal, responsible only for bootstrapping and setting up dependencies.

/config

Stores configuration files and setup logic, such as loading environment variables or external configuration. This package can also define structures for application configuration.

/internal

This is where the core logic of the application resides, split into packages based on functionality. Go restricts access to internal packages from external modules, keeping these packages private to the application. Each package (e.g., user, order) is self-contained, with its own models, services, and repositories. This is key to Go’s philosophy of encapsulation without excessive layering.

/internal/user – Manages all user-related functionality, including models (data structures), service (business logic), and repository (database interaction). This keeps user-related logic in one package, making it easy to maintain.
/internal/order – Similarly, this package encapsulates order-related code. Each functional area has its own models, services, and repositories.

/pkg

pkg holds reusable components that are used across the application but aren’t specific to any one package. Libraries or utilities that could be used independently, such as auth for authentication or logger for custom logging, are kept here. If these packages are particularly useful, they can also be extracted to their own modules later on.

/api

The API package serves as the layer for HTTP or gRPC handlers. Handlers here handle incoming requests, invoke services, and return responses. Grouping handlers by API version (e.g., v1) is a good practice for versioning and helps keep future changes isolated.

/utils

General-purpose utilities that aren’t tied to any specific package but serve a cross-cutting purpose across the codebase (e.g., date parsing, string manipulation). It’s helpful to keep this minimal and focused on purely utility functions.

Example Code Layout for the `user` Package

To illustrate the structure, here’s a closer look at what the user package might look like:

models.go

// models.go - Defines the data structures related to users

package user

type User struct {
    ID       int
    Name     string
    Email    string
    Password string
}

service.go

// service.go - Contains the core business logic for user operations

package user

type UserService struct {
    repo UserRepository
}

// NewUserService creates a new instance of UserService
func NewUserService(repo UserRepository) *UserService {
    return &UserService{repo: repo}
}

func (s *UserService) RegisterUser(name, email, password string) error {
    // Business logic for registering a user
    newUser := User{Name: name, Email: email, Password: password}
    return s.repo.Save(newUser)
}

repository.go

// repository.go - Defines database interaction methods for users

package user

type UserRepository interface {
    Save(user User) error
    FindByID(id int) (*User, error)
}

type userRepositoryImpl struct {
    // database connection or ORM
}

func NewUserRepository() UserRepository {
    return &userRepositoryImpl{}
}

func (r *userRepositoryImpl) Save(user User) error {
    // Logic to save user to database
    return nil
}

func (r *userRepositoryImpl) FindByID(id int) (*User, error) {
    // Logic to retrieve a user by ID from the database
    return &User{}, nil
}

Why This Package-Based Structure is Ideal for Go

This structure aligns well with Go’s idioms:

Encapsulation

By organizing packages based on functionality, the code is naturally encapsulated and modular. Each package owns its models, services, and repositories, keeping the code cohesive and highly modular. This makes it easier to navigate, understand, and test individual packages.

Minimal Interfaces

Interfaces are only used at the package boundaries (e.g., UserRepository), where they make the most sense for testing and flexibility. This approach reduces the clutter of unnecessary interfaces, which can make Go code harder to maintain.

Explicit Dependency Injection

Dependencies are injected via constructor functions (e.g., NewUserService). This keeps dependencies explicit and avoids the need for complex dependency injection frameworks, staying true to Go’s simplicity-focused design.

Reusability in /pkg

Components like auth and logger in the pkg directory can be shared across packages, promoting reusability without excessive coupling.

Clear API Structure

By grouping handlers under /api, it’s easy to scale the API layer and add new versions or handlers as the application grows. Each handler can focus on handling requests and coordinating with services, keeping the code modular and clean.

This package-centric structure lets you scale as you add more domains (e.g., product, inventory), each with its own models, services, and repositories. The separation by domain aligns with Go’s idiomatic way of organizing code, staying true to simplicity and clarity over rigid layering.

Opinions and Real-World Experiences

In my experience working with Go, Clean Architecture often complicates the codebase without adding significant value. Clean Architecture tends to make sense when building large, enterprise-grade applications in languages like Java, where there’s a lot of built-in support for DI, and managing deep inheritance structures is a common need. However, Go’s minimalism, its simplicity-first mindset, and its straightforward approach to concurrency and error handling create a different ecosystem altogether.

Conclusion: Embrace Go’s Idiomatic Architecture

If you’re coming from a Java background, it might be tempting to apply Clean Architecture to Go. However, Go’s strengths lie in simplicity, transparency, and modularity without heavy abstraction. An ideal architecture for Go prioritizes packages organized by functionality, minimal interfaces, explicit DI, realistic testing, and adapters for flexibility.

When designing a Go project, look to real-world examples like Kubernetes, Vault and the Golang Standards Project Layout. These showcase how powerful Go can be when the architecture embraces simplicity over rigid structure. Rather than trying to make Go fit a Clean Architecture mold, embrace an architecture that’s as straightforward and efficient as Go itself. This way, you’re building a codebase that’s not only idiomatic but one that’s easier to understand, maintain, and scale.

Is the Future Open Infrastructure? Why Cloud Giants Like AWS, GCP, and Azure Aren’t the Answer They Seemed to Be

Lucas de Ataides — Mon, 16 Sep 2024 22:58:59 +0000

Introduction

When cloud giants like AWS, Google Cloud (GCP), and Microsoft Azure first appeared, they promised businesses a revolution: lower costs, seamless scalability, and infrastructure that just works. No more spending big on hardware or dealing with maintenance headaches—just pay as you go, scale when you need to, and let the cloud handle the rest.

At first glance, it seemed perfect. Companies jumped on board, excited by the idea of turning capital expenses into manageable operational expenses. But now, as more businesses grow and dig deeper into the numbers, many are realizing these platforms aren't quite the cost-savers they were marketed as. Costs are soaring, and flexibility—well, it’s not as easy to achieve as the cloud pitch initially suggested.

In this landscape, open infrastructure is stepping up as a real alternative. It doesn’t just promise cost savings—it offers freedom from vendor lock-in and gives businesses the control they need to build systems that suit their needs, not the cloud providers’. Let’s explore why the big public clouds are falling short and how open infrastructure is positioning itself as the future of IT.

The Promise of Public Cloud: Reality Check

When AWS, GCP, and Azure first launched, they made bold promises:

Lower Costs: No need to buy expensive servers or keep data centers running.
Scalability: Quickly adjust resources up or down as your business needs.
Simplicity: A one-stop shop for everything you need—compute, storage, networking, and more.

For a while, it seemed like this was true. Startups and enterprises loved the idea of spinning up services without huge investments. But as businesses grew and started using more cloud resources, the story changed. What initially seemed like savings turned into unpredictable bills, and suddenly, that flexibility didn’t feel quite so easy.

Why Public Cloud Isn’t as Cheap as It Looks

Confusing Pricing: One of the biggest complaints about public cloud services is their incredibly complex pricing models. At first, you might be paying for just a few services, and everything seems affordable. But as you expand—adding more instances, storage, or specific services—costs quickly spiral out of control. Many companies find that they’re regularly exceeding their budgets, especially once they realize how much they’re paying for things like data transfer and premium services.
Vendor Lock-in: The deeper you go into a cloud provider’s ecosystem, the harder it is to leave. Services like AWS Lambda or Google BigQuery are convenient, but they’re built with proprietary tools that make moving to another cloud provider an expensive, time-consuming process. Once your infrastructure is tied to their services, the cost of migrating elsewhere can be staggering.
Hidden Data Transfer Costs: One of the sneaky costs of the public cloud is egress fees—the charges for moving data out of the cloud. These fees aren’t always obvious at first but can quickly add up, especially for companies that deal with large amounts of data. For example, if you’re regularly transferring data between clouds or back to on-premises systems, you might find that your transfer fees eclipse any savings you thought you’d gain from using the cloud in the first place.
Expensive Proprietary Services: While the basic offerings of cloud providers might seem affordable, it’s the specialized services that can break the bank. Solutions like AWS RDS or GCP’s BigQuery are powerful but come with steep pricing. As your business grows and you become more reliant on these services, the cost-benefit of using the cloud starts to fade, and moving away from them becomes a complex ordeal.

Open Infrastructure: A Better Path Forward

So, if the big public clouds aren’t delivering the cost savings and flexibility they promised, what’s the alternative? Enter open infrastructure, a way of building cloud environments that gives businesses more control, transparency, and freedom.

Open infrastructure is all about using open-source projects and standards to build your own cloud environment, whether in your private data center, a public cloud, or a hybrid model that uses both. Some of the most important technologies in this space include:

OpenStack: An open-source cloud computing platform that lets you run your own data center just like the public cloud, but with more control over your resources and data.
*StarlingX: A cloud infrastructure software stack for the edge used by the most demanding applications. It provides a container-based infrastructure for edge implementations in scalable solutions that is ready for production.
Kubernetes: A popular open-source platform for running distributed applications that can work across any cloud or on-premises infrastructure, offering true flexibility.
Ceph: An open-source solution for scalable, distributed storage that provide a real alternative to the high-priced options offered by proprietary cloud services.

Open infrastructure lets companies build their systems exactly how they need them without being locked into any one provider. This means more choice, more control, and ultimately, more freedom.

Why Open Infrastructure is the Better Choice

Transparent Costs: With open infrastructure, you’re paying for what you use in a more straightforward way. There are no hidden fees or surprise bills. You have full visibility into your hardware and network costs and can control exactly how and where resources are deployed.
No Vendor Lock-in: Because open infrastructure uses open standards, you’re not tied to any single cloud provider’s tools or services. This gives you the flexibility to move workloads between environments, whether that’s across different cloud providers or back on-premises, without needing to completely rebuild your systems.
Hybrid and Multi-Cloud Flexibility: Open infrastructure makes it easy to build systems that work across multiple clouds or mix on-premises and cloud environments. With technologies like Kubernetes, you can move workloads across different environments as needed, optimizing both performance and cost while maintaining control over your architecture.
Community-Driven Innovation: Open-source projects are powered by communities of developers from around the world. This means open infrastructure solutions are continuously evolving, improving, and adapting to the latest needs of businesses—without being driven solely by the profit motives of big tech companies.

The Open Infrastructure Movement is Growing

The 2024 State of Open Infrastructure report shows that more and more companies are moving toward open solutions, realizing the flexibility, innovation, and cost transparency they offer. Businesses across industries, from telecom to finance, are adopting hybrid and multi-cloud strategies powered by open technologies, finding that open infrastructure is not just a cost-saving measure but a strategic advantage.

The Open Infrastructure Foundation is leading this charge, laying out a blueprint for how open technologies can provide a scalable, reliable alternative to the traditional cloud model. Organizations that embrace open infrastructure aren’t just avoiding the downsides of public cloud—they’re also positioning themselves to innovate more quickly and adapt more easily to changing business needs.

The Future is Open - Are You Ready?

The cloud was supposed to bring lower costs, scalability, and simplicity, but for many businesses, it’s delivered the opposite. Open infrastructure is offering a way out—giving companies the control, flexibility, and cost transparency they’ve been looking for all along.

The future of IT doesn’t have to be tied to the proprietary ecosystems. It’s about building open, flexible systems that put control back in your hands.

The future is open infrastructure.