loading...

Flowcharts for incident response

lbonanomi profile image lbonanomi ・1 min read

Does this sound familiar?

The alarms are flashing.
The clients are emailing.
Upper management is calling.
Line managers are DM-ing.

You are trying to juggle all of this while trying to follow a 50-line paragraph of troubleshooting text...

Let's borrow the Graphviz tools from the systems architect to make easier-to-follow incident response plans.

digraph Jira_Outage 
{
  {
    first_notification [ 
      label="Notification of a Jira outage from any channel" 
      shape=rect
    ] 

    pc_browser [ 
      label="Does Jira load in a PC Browser?" 
      shape=rect 
    ]
    yes_pc_browser [ 
      label="Yes" 
      shape=rect 
    ]

    no_pc_browser [ 
      label="No" 
      shape=rect 
    ]

    linux_server_browser [ 
      label="Does Jira load in\nFirefox from a Linux server?" 
      shape=rect 
    ]

    user_ssh_connect [
      label="Does SSH-ing to jira.internal.hostname connect?" 
      shape=rect 
    ]

    yes_user_ssh_connect [ 
      label="Yes"
      shape=rect
    ]

    no_user_ssh_connect [
      label="No"
      shape=rect 
    ]

    debug_screen [
        label="Jira shows an unexpected screen."
        shape=rect
    ]

    pingable [
      label="Does jira.internal.hostname ping?"
      shape=rect
    ]

    host_down [
      label="Application host is down.\n\n\Attempt to restart application."
      shape=rect
    ]

    yes_ping [
      label="Yes"
      shape=rect
    ]

    no_ping [
      label="No"
      shape=rect
    ]

    site_down [
      label="Site is down.\n\nSwitch to backup site."
      shape=rect
    ]

    first_notification -> pc_browser

    pc_browser -> no_pc_browser
    pc_browser -> yes_pc_browser 
    no_pc_browser -> user_ssh_connect
    yes_pc_browser -> debug_screen

    user_ssh_connect -> no_user_ssh_connect
    user_ssh_connect -> yes_user_ssh_connect

    yes_user_ssh_connect -> linux_server_browser
    no_user_ssh_connect -> pingable

    pingable -> no_ping
    pingable -> yes_ping

    yes_ping -> host_down
    no_ping -> site_down

    linux_server_browser -> site_down
    debug_screen -> host_down

  }
}

into handsome instructional flowcharts like this:

I'm sure you're wondering why I prefer Graphviz to any of the plethora of free SVG diagramming tools. Besides fundamental masochism, using dot allows for automatic document (re)generation in the face of hosts being added/deleted/changed.

Posted on by:

lbonanomi profile

lbonanomi

@lbonanomi

Internet loudmouth since 1996

Discussion

markdown guide