DEV Community: Jan Pokorný

Terraform finally met switches: Cisco Network as Code with ACI just works

Jan Pokorný — Thu, 29 Oct 2026 08:45:50 +0000

Every network engineer wants to automate their network. Automation is our pride. Working automation is proof of good architecture — and in large setups, an operational necessity. Yet I've always felt that switch vendors aren't exactly helping us.

Networking is a conservative world. Every time I dove into automation, I ended up frustrated by how far behind it is. First, there are the archaic protocols that nobody truly understands and that — at least everywhere I've worked — never delivered in practice: SNMP in write mode, NETCONF, RESTCONF, screen-scraping over SSH. (Yes, OpenConfig and gNMI exist; if your vendor mix supports them well, I envy you.) Right next to them stands a long parade of revolutionary proprietary management systems from the vendors themselves. The trouble with those is always the same: they're heavyweight, they integrate poorly with anything else, and above all they have the lifespan of a mayfly. Just for HP switches, I'm running out of fingers on one hand counting how many times their primary management tool has changed.

Ansible and the REST APIs appearing on switches looked like a small ray of hope. But even there, disappointment came with time. Ansible modules typically covered only some parts of the configuration — full coverage was never on the table. Partly because modules were missing, partly because Ansible simply isn't the right tool for this domain: there's no state, no plan/diff before you push, and deleting things is entirely on you — which is exactly what makes it unmaintainable at scale. The final nail in the coffin is opening GitHub and seeing that the last commit to the relevant collection was five years ago.

So in the end, we all land back at SSH and pushing CLI commands. Hand on heart — isn't that so? Thanks to netmiko we can more or less reliably shove config onto switches and call it automation. But when someone asks whether we can also change a config, or — heaven forbid — safely delete one, we change the subject.

From Ansible fan to Terraform convert

I used to be a big Ansible fan. I built some genuinely nice, complex projects with it. I even wrote my own modules. Then I got pulled into a cloud project that required Terraform. The moment I tried it, I understood: for infrastructure, Terraform is the hammer. And from that moment on, every problem I saw was a nail. From day one I kept thinking: I wish I could manage switches like this.

Then the opportunity came. We decided to upgrade the network in our datacenters, and Cisco ACI was one of the contenders. I noticed Cisco offers Terraform-based management, so I tested it during the PoC. It worked well, and the conclusion was that it could cover our day-to-day operations and unlock automation.

Long story short: today, 99% of our entire ACI configuration is driven by Terraform. From NTP to L3Outs.

What Cisco Network as Code actually is

Cisco NaC is essentially ready-to-eat Terraform, prepared directly by Cisco. And credit where it's due — Cisco did a phenomenal job here (at least for ACI). We run it on ACI 6.1(5e) with NaC module version 2.0.0.

Why does it work so well? In my view, two cornerstones:

1. APIC. Managing one central controller with a high-quality API is orders of magnitude easier than managing dozens or hundreds of individual switches. Terraform talks to a single API with one coherent view of the fabric, not to a fleet of boxes.

2. Separation of data from code. As a user, you practically don't need to know Terraform at all. You don't write Terraform — you write a data structure (YAML), which the Terraform module reads and assembles itself into whatever is needed. This isn't a Cisco invention — Google does something similar in their Fabric FAST framework for landing zones — but Cisco executed it really well. A complete beginner can have a working project within a few hours.

Look how simple the Terraform part is. This is essentially the whole thing:

terraform {
  required_providers {
    aci = {
      source = "CiscoDevNet/aci"
    }
  }
}

provider "aci" {
  username = var.aci_user
  password = var.aci_password
  url      = var.aci_url
}

module "aci" {
  source  = "netascode/nac-aci/aci"
  version = "2.0.0"

  yaml_directories = ["data/**"]

  manage_access_policies    = true
  manage_fabric_policies    = true
  manage_pod_policies       = false
  manage_node_policies      = true
  manage_interface_policies = true
  manage_tenants            = true
}

It really just says how to connect to the APIC, which configuration sections Terraform should manage, and where to load the data model from. You flip on only the sections you want Terraform to own — we leave pod policies at their day-0 defaults, which is pretty much the missing 1 % from the claim above.

The data model itself is plain YAML. A bridge domain looks like this:

---
apic:
  tenants:
    - name: company
      bridge_domains:
        - name: d2-dev-lan-1_1201
          vrf: dc2-l3-fix
          unicast_routing: false
          description: General LAN network

And the best part: you can split the data model into as many small files as you like and build any directory structure that suits your team. Terraform merges everything into one big YAML document and processes that. Our structure looks like this:

├── docs
├── iac
│   └── data
│       ├── access_policies
│       │   ├── interface_policies
│       │   └── switch_policies
│       ├── fabric_policies
│       ├── interface_policies
│       │   ├── 101_102
│       │   ├── 103_104
│       │   ├── 105_106
│       │   └── ...
│       ├── node_policies
│       └── tenants
│           ├── mgmt
│           ├── company
│           │   ├── application_profiles
│           │   │   └── endpoint_groups
│           │   ├── bridge_domains
│           │   └── vrfs
│           └── lab
└── scripts

Cisco's model covers, I dare say, 100% of the configuration — I haven't yet hit anything I couldn't find — and the documentation is perfectly readable.

Terraform is not the automation. It's the key to it.

Managing APIC with Terraform is not automation by itself. It's the key that unlocks it.

It unlocks GitOps. Your complete configuration is now text, living in git. You have every change and the complete history — from the day the fabric was born to the last update. You know who made each change and based on what. (We had the luxury of a greenfield deployment — our fabric was born straight into git. Adopting NaC on an existing brownfield fabric is possible, but it's a story of its own.)

It unlocks CI/CD and real change management. Suddenly, changes aren't made by a specific engineer clicking in a GUI — they're made by a pipeline. Every change is a pull request that someone has to approve. In the PR you see exactly what is being proposed, and exactly what Terraform plans to do on the APIC side (terraform plan is posted right into the PR as a comment). Review rules are explicit and enforced. And before anything gets applied, the pipeline takes an APIC configuration snapshot. No snapshot, no apply. Period.

NetBox as the source of truth

Now for the actual automation. We record absolutely everything in NetBox — it is our source of truth. Devices, every single cable, VLANs, prefixes, IP addresses, VRFs, virtualization clusters… everything is diligently documented there.

Since the ACI side is now just YAML files, nothing was easier than adding a few custom fields to NetBox and — with a handful of prompts to Claude — building a script that pulls data from the NetBox API and updates the YAML files. New network segment? Add a record to the central documentation, and within minutes there's a pull request waiting for approval. Approve, and it's deployed.

Or skip the scripts — bring an AI agent

Maybe writing Python scripts isn't trendy anymore. Maybe it isn't even necessary. The next step is an AI agent. If there's one thing AI agents are unbeatable at, it's working with text — and what better task could one get than generating YAML from a work order?

"I've added server company-server-01 to NetBox. Prepare the configuration to connect it to ACI."

The agent knows NetBox exists. It can look up which rack the server is in and which leaf switches live in that rack. It can check what cable type is planned. It knows our port-allocation rules and the ACI object naming convention. This is a trivial task — even a small model handles it comfortably. And there's nothing to be afraid of: it does exactly the same thing a human would. It opens a pull request for approval. There's still a human in the loop.

The risks (and how to handle them)

It wouldn't be fair to end without the downsides. Cisco NaC has a few, and they're manageable.

The growing state file

Every new section of configuration means a bit more time for Terraform to compute its plan. In the beginning, Terraform was snappy; as the project grows, the full cycle takes longer and longer. This isn't an ACI or NaC problem — it's a classic Terraform problem, and the solution is classic too: split the project into sub-projects per configuration section. If I'm working on access policies, there's no need to recompute state for tenants, and vice versa — and when both change, they can run in parallel. Cisco NaC already divides the configuration tree into six sections (you can see them as the manage_* flags in main.tf above), which is the natural place to cut. Splitting below that level feels risky to me. We're not that big and I'm confident we'll manage all-in-one — but for huge setups, this would be a real topic.

Drift — and the blast radius of an apply

The elephant in the room: what happens when someone changes something in the GUI? Terraform doesn't know about it — until the next plan, which flags it as drift, and the next apply, which reverts it. That's either your best friend or a production incident, depending on whether the GUI change was a mistake or an emergency fix someone forgot to backport.

We handle it with two rules. First, configuration changes go through pull requests only — the GUI is for reading and troubleshooting. Second, the pipeline runs terraform plan on a schedule, so any drift shows up as an alert within hours, not months later during an unrelated change.

The same mechanism cuts the other way, too: a bad PR can produce a plan that destroys objects — and on a production fabric, X to destroy can mean EPGs going away along with the traffic in them. This is exactly why the plan is posted into the PR: the reviewer sees the add/change/destroy counts before approving, and the pre-apply APIC snapshot is the last resort. An automated guard that blocks plans with unexpected destroy counts would be a sensible next hardening step — reviewers are human.

No referential integrity between objects

Thanks to the data/code separation, the links between objects are gone. Nothing stops you from referencing a policy that doesn't exist — typically a typo in a name. With hand-written Terraform and its resource references, this couldn't happen. It's a small thing, but worth knowing. There's a possible fix: nac-validate, a CLI that validates the YAML against a schema and runs semantic rules (written in Python) that should catch dangling references before the plan even runs. I'll be honest — I only gave it a quick try, it didn't really grab me, and we don't use it for now. Your mileage may vary, so it's worth a look. Until then, an AI agent doing a pre-push textual consistency check finds this kind of problem effortlessly, and worst case, the APIC will raise a fault and tell you anyway.

What if it all falls apart?

What if, for whatever reason, you can't continue with Terraform? Then nothing stops you from simply throwing it away and continuing another way. The objects Terraform created are ordinary ACI objects — you just start managing them differently. There's no lock-in in the data. Support-wise, one nuance worth knowing: the underlying ACI Terraform provider is fully TAC-supported, while the NaC modules themselves are open source maintained by Cisco, with Cisco Professional Services behind them for customization.

Closing thoughts

For years, "network automation" meant heroically wrapping duct tape around SSH. With ACI and Cisco Network as Code, for the first time I feel like we're doing infrastructure-as-code on the network with the same comfort the cloud folks have had for a decade: declarative data model, git history, PR-based change management, CI/CD, and a source of truth feeding it all.

If you're running ACI and still clicking in the GUI — try the PoC. It took us a few hours to get the first working project, and we never looked back.

Happy to answer questions about our setup — pipelines, NetBox integration, or the AI agent workflow. Drop them in the comments.

No SNMP OIDs for DOM stats on Cisco ACI switches? Use the API and Zabbix.

Jan Pokorný — Wed, 05 Aug 2026 07:57:00 +0000

Tested on Zabbix 7.0.0 and Cisco ACI 6.1(5e).

If you run optical links, you probably already know this: the moment you actually need DOM stats (Digital Optical Monitoring — the light levels your SFPs see and send) is the moment it's too late to start collecting them. The value is in the history.

On our previous infrastructure we collected SFP Rx/Tx power over SNMP as a matter of course, and it paid off more than once. Watching the values over time makes troubleshooting flapping links much less of a guessing game — and in one case it did something even better: we had a batch of SFPs from a bad production series, and the slow, steady decline in their optical levels let me predict the failures about a week before they actually died. Replacing optics on your own schedule instead of at 3 AM is a good trade for a few Zabbix items.

Then we moved to Cisco ACI, and the SNMP approach hit a wall.

The problem: ACI doesn't expose DOM stats over SNMP

As far as I can tell, Cisco ACI switches (Nexus 9000 series in ACI mode) simply don't have SNMP OIDs for DOM statistics. I tested this myself, walking the usual suspects, and found nothing. I even gave an Anthropic Opus model access to SNMP on the switches and let it dig through the MIBs on its own — same conclusion: it's just not there.

I'm honestly not sure whether this is a limitation of the switch series or of ACI mode specifically. My money is on ACI mode, because on our Inter-Pod Network — similar but different Cisco Nexus switches running in NX-OS mode — the DOM OIDs exist and work exactly as you'd expect. Same vendor, same optics concept, different operating mode, completely different story.

DOM stats are collected — by the APIC (once you turn it on)

The good news: the APIC controller can collect DOM statistics from the whole fabric. The less good news: for some reason it's not enabled by default. You have to switch it on here:

Fabric → Fabric Policies → Policies → Monitoring → Fabric Node Controls → Enable DOM

Why isn't this the default? Genuine question. It's cheap telemetry about the physical layer of every link in your fabric, the hardware is already measuring it, and the failure mode it catches is one of the most common ones in a data center. If anyone from the BU is reading this — I'd love to hear the reasoning.

Once enabled, the APIC collects the values, and I believe it would raise a fault if a value crossed the configured thresholds. But what the APIC can't do is draw you a graph of those values over time. And as I said at the start, the history is the whole point — a single snapshot won't tell you that an SFP has been quietly losing 0.2 dB a week. So we want these values in an external monitoring system.

Monitoring over the API instead

No SNMP, but there's a perfectly good REST API. The catch: classic NMS tools and REST APIs don't get along very well. With Zabbix, though, this is very doable.

The typical pain point of API-based monitoring is authentication. It's usually a multi-step dance: first a login call to get a token, then the actual query using that temporary token, plus refresh logic when it expires. That's exactly the kind of statefulness you don't want in a monitoring item — ideally, every poll should be a single self-contained API call.

Cisco APIC has a neat answer to this: certificate-based (signature) authentication. You generate an X.509 certificate, attach it to a local APIC user, and then sign each request with the private key. The signature travels in a Cookie header alongside the request itself — no login, no token, no session, one HTTP call.

It's slightly more involved to set up, because you have to generate the certificate and sign every request. But it solves the one-call problem, and there's a nice security bonus: since a signature is only valid for the exact request it was computed for, your monitoring system effectively gets access to a handful of specific API calls with specific attributes — and nothing else. Even if someone extracts the signed cookie from Zabbix, all they can do with it is read DOM stats. Then you just parse the JSON response and you're done.

Here's how to set the whole thing up, end to end.

Step 1: Generate a certificate

A plain self-signed cert is fine. Ten years of validity so you don't have to think about it again any time soon (after 10 years, generate and upload a new one):

openssl req -newkey rsa:2048 -nodes \
  -keyout zabbix-monitoring.key \
  -x509 -days 3650 \
  -out zabbix-monitoring.crt \
  -subj "/CN=zabbix-monitoring"
chmod 600 zabbix-monitoring.key

Keep the .key file private — it never leaves the machine that signs requests. The .crt is the public part that goes to APIC.

If you'd rather not do this by hand, there's a ready-made script in my GitHub repo (more on that at the end) that generates the pair and prints everything you'll need for the next steps.

Step 2: Create the APIC user and attach the certificate

In the APIC GUI:

Admin → AAA → Users → Create Local User, e.g. zabbix-monitoring.
Assign the correct permissions. I used security domain all, role read-all, with Read privilege.
Open the user, go to X.509 Certificates, click +, name the certificate (I keep it identical to the username, e.g. zabbix-monitoring), and paste in the contents of the .crt file.
Save.

Two values from this step matter later:

The certificate DN, which follows the pattern uni/userext/user-<username>/usercert-<certname>. With the names above: uni/userext/user-zabbix-monitoring/usercert-zabbix-monitoring
The certificate's SHA256 fingerprint, which you can get locally:

openssl x509 -in zabbix-monitoring.crt -noout -fingerprint -sha256

Step 3: The URL, and how to sign a request

DOM stats live in two APIC object classes — one for the receive side, one for the transmit side:

Rx: /api/class/ethpmDOMRxPwrStats.json
Tx: /api/class/ethpmDOMTxPwrStats.json

Important detail: Rx and Tx are two separate API calls, and therefore two separate signed requests. The signature covers method + URI + body of one specific request, so you can't reuse the Rx cookie for the Tx endpoint. You'll go through the signing below twice, once per URI.

The signing itself is: take the string GET + the URI (path and query string, no host), sign it with your private key (SHA256 + PKCS#1 v1.5), and base64 the result. In plain shell:

URI='/api/class/ethpmDOMRxPwrStats.json'

SIGNATURE=$(printf 'GET%s' "$URI" \
  | openssl dgst -sha256 -sign zabbix-monitoring.key \
  | base64 -w 0)

FINGERPRINT=$(openssl x509 -in zabbix-monitoring.crt -noout -fingerprint -sha256 \
  | sed 's/^.*Fingerprint=//')

DN='uni/userext/user-zabbix-monitoring/usercert-zabbix-monitoring'

COOKIE="APIC-Certificate-Algorithm=v1.0; APIC-Certificate-Fingerprint=$FINGERPRINT; APIC-Certificate-DN=$DN; APIC-Request-Signature=$SIGNATURE"

echo $COOKIE

And the test call:

curl -sk "https://<your-apic>$URI" -H "Cookie: $COOKIE"

If everything is wired up correctly, you get back JSON along these lines (trimmed):

{
  "totalCount": "214",
  "imdata": [
    {
      "ethpmDOMRxPwrStats": {
        "attributes": {
          "dn": "topology/pod-1/node-101/sys/phys-[eth1/1]/phys/domstats/rxpower",
          "value": "-2.334269",
          "hiAlarm": "1.999912",
          "hiWarn": "0.000000",
          "loAlarm": "-13.695722",
          "loWarn": "-10.300390",
          "alert": "none"
        }
      }
    }
  ]
}

One object per optical interface in the whole fabric, with the current Rx power in dBm plus the module's own alarm/warning thresholds. The dn tells you exactly which pod/node/port it is.

For Tx, repeat the exact same procedure with URI='/api/class/ethpmDOMTxPwrStats.json' — nothing else changes.

There's also a Python script in the repo that does the signing for you and prints the finished Cookie header value:

./generate-apic-cookie.py \
  --key zabbix-monitoring.key \
  --cert zabbix-monitoring.crt \
  --dn "uni/userext/user-zabbix-monitoring/usercert-zabbix-monitoring" \
  --uri "/api/class/ethpmDOMRxPwrStats.json"

Step 4: Wiring it into Zabbix

Now the Zabbix side. The pattern is: one HTTP agent item fetches the raw JSON, a low-level discovery rule finds all the interfaces in it, and dependent item prototypes slice out the per-interface dBm values — so the APIC gets polled exactly once per cycle no matter how many optics you have.

4.1 Create the host and macros

Create a host for your APIC (Data collection → Hosts → Create host) and give it an interface with the APIC's address. The interface type doesn't matter here — it's just a holder for the APIC's FQDN/IP (used via {HOST.CONN} below), since the actual polling happens through HTTP agent items, not the interface itself. Then add two host macros holding the full Cookie header values you generated in step 3:

{$APIC.COOKIE.RX} = APIC-Certificate-Algorithm=v1.0; APIC-Certificate-Fingerprint=...; APIC-Certificate-DN=...; APIC-Request-Signature=...
{$APIC.COOKIE.TX} = same idea, generated for the Tx URI

4.2 The raw HTTP agent item

Create an item on the host:

Name: APIC Rx power raw
Type: HTTP agent
Key: apic.optics.rx.raw
URL: https://{HOST.CONN}/api/class/ethpmDOMRxPwrStats.json
Headers: Cookie → {$APIC.COOKIE.RX}
Type of information: Text
History / Trends: Do not store (this item only exists to feed the items below — no point storing the huge JSON blob)

Test it (the Test button in the item form) — you should see the same JSON as from curl.

4.3 The discovery rule

Create a discovery rule on the host:

Name: APIC Rx optics discovery
Type: Dependent item, Master item: apic.optics.rx.raw
Key: apic.optics.rx.discovery
Preprocessing: one step, JSONPath → $.imdata
LLD macros: {#DN} → $.ethpmDOMRxPwrStats.attributes.dn

This turns every interface in the JSON into a discovered entity identified by its DN.

4.4 The item prototype

Inside the discovery rule, create an item prototype:

Name: DOM stats - Rx dbm [{#DN}]
Type: Dependent item, Master item: apic.optics.rx.raw
Key: apic.optics.rx.dbm[{#DN}]
Type of information: Numeric (float), Units: dbm
Preprocessing: two JSONPath steps:
1. $.imdata[?(@.ethpmDOMRxPwrStats.attributes.dn=="{#DN}")] — pick this interface's object out of the array
2. $[0].ethpmDOMRxPwrStats.attributes.value — extract the dBm value

And that's it. After the next discovery run, Latest data fills up with one Rx power item per optical interface in the fabric, and Zabbix starts building the history that started this whole exercise.

For Tx, clone the whole chain (raw item → discovery rule → prototype), swap the URL to ethpmDOMTxPwrStats.json, the cookie macro to {$APIC.COOKIE.TX}, and RxPwrStats to TxPwrStats in the JSONPaths.

The response also carries each module's own hiAlarm/hiWarn/loAlarm/loWarn thresholds, so you can build alerting on top of this — evaluating the value against those limits and firing triggers. I've kept that out of this article to keep it focused, but a finished version of exactly that (a status item computed in JavaScript preprocessing plus warning/critical trigger prototypes) is in the GitHub template.

The repo

While building this, I went looking for an existing Zabbix template for Cisco APIC — and came up empty. So I made one and put it on GitHub:

github.com/jpokorny-cz/zabbix-cisco-aci-templates

If you'd rather skip the manual setup above entirely, you can just import the template — everything from this article is in there, plus a good deal more: fabric health score, fault counts by severity, controller status, CPU/memory/disk usage, threshold evaluation and triggers for the optics, and a companion template for Nexus switches in the Inter-Pod Network role. The cert and cookie scripts I linked are in there too.

If you run ACI and Zabbix, give it a try — and if you extend it or find something broken, issues and pull requests are very welcome. And if you've solved ACI DOM monitoring some other way (streaming telemetry? something I missed?), I'd genuinely like to hear about it in the comments.