Code Your Diagrams: Automate Architecture with Python's Diagrams Library

Introduction

In the realm of modern infrastructure, where cloud services and microservices reign supreme, managing and visualizing complex architectures is more critical than ever.

Gone are the days of manually creating and updating architecture diagrams. With the Diagrams Python library, you can generate dynamic, code-driven diagrams that evolve alongside your infrastructure. A few lines of Python are all it takes to visualize cloud architectures, network topologies, or microservice interactions. Diagrams ensure your system documentation stays accurate and up-to-date, whether you’re managing multi-cloud deployments, Kubernetes clusters, or on-premise solutions. It’s an effortless way to keep your architecture in sync with your codebase.

In this post, we'll explore the capabilities of the Diagrams library, showcase how to create High-Level Designs (HLDs) for cloud infrastructure, and automate the process of creating architecture diagrams.

Why Use Diagrams Python Library?

• Automation: Generate architecture diagrams directly from your code, ensuring they stay up-to-date with the evolving system.
• Programmatic Control: Diagrams allows you to define your infrastructure visually with Python, offering fine control over how elements are represented.
• Supports Multiple Cloud Providers: The library supports AWS, Azure, GCP, and on-premise systems, making it a versatile tool for visualizing multi-cloud and hybrid architectures.
• Scalable: From small projects to large distributed systems, Diagrams can handle various levels of complexity.

Supported Providers

• OnPrem
• AWS
• Azure
• GCP
• IBM
• Kubernetes (K8s)
• AlibabaCloud
• OCI (Oracle Cloud Infrastructure)
• OpenStack
• Firebase
• DigitalOcean
• Elastic
• Outscale
• Generic
• Programming
• SaaS
• C4 Model
• Custom

Getting Started with Diagrams

To get started with Diagrams, you’ll need to install the library and set up your environment.

Step 0: Prerequisites

To create diagrams using the Diagrams Python library on MacOS/Windows, you'll need to install Graphviz first. Graphviz is the tool that the Diagrams library uses to generate the visual representations of your infrastructure.

• Mac

If you're on macOS, the easiest way to install Graphviz is using Homebrew:

brew install graphviz

• Windows

If you're on Windows, follow the below steps

1. Download the Graphviz installer from the official website Graphviz Download Page.
2. Run the installer and follow the installation steps.
3. During installation, make sure to check the option that adds Graphviz to your system’s PATH.

Step 1: Installation

pip install diagrams

Step 2: Your First Diagram

Let’s create a simple diagram that represents a basic web architecture on AWS.

from diagrams import Diagram
from diagrams.aws.compute import EC2
from diagrams.aws.network import ELB
from diagrams.aws.database import RDS

with Diagram("Simple AWS Architecture", show=False):
    lb = ELB("Load Balancer")
    web = EC2("Web Server")
    db = RDS("Database")

    lb >> web >> db

With this minimal code, you can visualize how traffic flows from the Load Balancer to the Web Server, and then to the Database. That’s the power of the Diagrams library: it's quick, intuitive, and highly customizable. And this is just the beginning—there are many more advanced features and components you can leverage, which we'll explore in the following sections.

Advanced Features

Grouping Components (Clustering)

You can group components into clusters to represent different tiers or logical groupings within your architecture.

from diagrams import Cluster, Diagram
from diagrams.aws.compute import EC2
from diagrams.aws.network import ELB
from diagrams.aws.database import RDS

with Diagram("AWS Architecture with Clustering", show=False):
    with Cluster("Web Tier"):
        lb = ELB("Load Balancer")
        web_servers = [EC2("Web 1"), EC2("Web 2")]

    with Cluster("Database Tier"):
        db_primary = RDS("Primary DB")
        db_replica = RDS("Replica DB")

    lb >> web_servers >> db_primary
    db_primary >> db_replica

We use Cluster() to group web servers and databases, making the diagram easier to understand by visualizing tiers separately.

Customizing Components

Diagrams allow you to add custom labels, colors, and even custom images to represent specific components. For example, if you want to represent a custom service, you can include external images from local or even from remote.

• Using a Custom Icon from a Local Source

If you have an icon saved locally (for example, a custom_icon.png file), you can use it to represent your custom component in the diagram. The code below shows how to add a custom icon from your local filesystem.

from diagrams.custom import Custom
with Diagram("Custom Service Architecture", show=False):
    custom_service = Custom("My Custom Service", "./custom_icon.png")

./custom_icon.png is the path to your local image file.

• Using a Custom Icon from a Remote Source

Similarly, you can use an image from a remote source. Here's how you can download an image from a URL and use it in your diagram.

You can also use custom icons from a remote URL by giving the remote path to the files.

from diagrams import Diagram, Cluster
from diagrams.custom import Custom
from urllib.request import urlretrieve

with Diagram("Custom with remote icons", show=False, filename="custom_remote", direction="LR"):

  # download the icon image file
  diagrams_url = "https://github.com/mingrammer/diagrams/raw/master/assets/img/diagrams.png"
  diagrams_icon = "diagrams.png"
  urlretrieve(diagrams_url, diagrams_icon)

  diagrams = Custom("Diagrams", diagrams_icon)

This allows for even more flexibility in designing architecture that suits your organization's needs.

Combining Multi-Cloud and On-Premise Architectures

We can also use a combination of on-premise systems and cloud infrastructure, Diagrams makes it easy to combine these elements into a single view. You can visualize hybrid architectures seamlessly.

from diagrams import Diagram, Cluster, Edge
from diagrams.aws.compute import EC2
from diagrams.aws.network import ELB
from diagrams.aws.database import RDS
from diagrams.gcp.compute import GCE
from diagrams.azure.compute import VM
from diagrams.onprem.compute import Server
from diagrams.onprem.client import User
from diagrams.generic.network import Firewall, Switch, Router
from diagrams.generic.storage import Storage  

with Diagram("Multi-Cloud and On-Prem Architecture", show=False):

    # On-premise infrastructure
    with Cluster("On-Premise Data Center"):
        users = User("Users")
        firewall = Firewall("Firewall")
        router = Router("Router")
        switch = Switch("Switch")
        onprem_server = Server("Local Server")
        storage = Storage("Storage")

        users >> firewall >> router >> switch >> onprem_server
        onprem_server >> storage

    # AWS infrastructure
    with Cluster("AWS Cloud"):
        aws_lb = ELB("Load Balancer")
        aws_ec2 = EC2("App Servers")
        aws_rds = RDS("Database")

        aws_lb >> aws_ec2 >> aws_rds

    # GCP infrastructure
    with Cluster("GCP Cloud"):
        gcp_gce = GCE("Compute Engine")
        gcp_storage = Storage("GCP Storage")

        gcp_gce >> gcp_storage

    # Azure infrastructure
    with Cluster("Azure Cloud"):
        azure_vm = VM("Virtual Machines")
        azure_storage = Storage("Azure Storage")

        azure_vm >> azure_storage

    # Connecting on-prem to cloud components via VPN
    router >> Edge(label="VPN") >> [aws_lb, gcp_gce, azure_vm]

Challenges and Limitations

Although Diagrams is a powerful tool, there are some challenges:

• Performance: Generating very large diagrams with hundreds of components can be slow.
• Customization Limitations: While Diagrams offers a wide range of predefined components, adding highly customized elements might require extra work.
• Static Output: Diagrams generate static images. If you need interactive or real-time diagrams, you may need to integrate them with other tools.

Conclusion

The Diagrams Python library is a fantastic tool for automating the creation of infrastructure diagrams. By integrating it into your workflows, you can generate architecture diagrams dynamically as your infrastructure changes. Whether you’re documenting your cloud infrastructure or illustrating complex microservice architectures, Diagrams offers a powerful, programmatic way to visualize your systems

GitHub Repository

You can find the complete source code for the examples in this blog on my GitHub:

• My Diagrams Code Repository

Reference

• Diagrams: https://diagrams.mingrammer.com/docs/getting-started/installation

Disclaimer:

This is a personal blog. The views and opinions expressed here are only those of the author and do not represent those of any organization or any individual with whom the author may be associated, professionally or personally.