Transible — represents cloud configuration as Ansible playbooks

What is Transible

Short description

This tool takes your current cloud configuration and represents it as Ansible playbooks. Its repository on GitHub is: https://github.com/sshnaidm/transible

More details

It takes servers, security rules, images, volumes and all other things your cloud includes and defines their config as tasks in Ansible playbooks that are ready for deployment. By running these playbooks you actually can deploy or redeploy your current cloud.
Simple run:

git clone https://github.com/sshnaidm/transible
cd transible
./transible.py --os-cloud your_cloud_name

Why would anybody redeploy its existing cloud?

Use-cases of Transible

Freezing cloud config for further IaaC management

There are cases when you have a configured cloud with a lot of servers, networks,
images, etc. But sometimes we don't have this properly managed with
Infrastructure as Code principles and cloud maintenance turns to be a nightmare.
Hundreds of legacy servers with unknown configurations, complex network setups that nobody knows who made them and for what purpose. Just collecting information from the cloud can be a complex task when it's under load and things change quickly. Removing legacy and things that look unused can cause breakages in the work of others, while it's almost impossible to rollback when it turns out this deleted server is in usage and very important for other teams.
It's pretty difficult and sometimes even impossible manually to create IaaC configs for existing infrastructure that can include hundreds of servers or more, Transible comes to solve this problem.

Transible can take your cloud and convert everything in it to Ansible playbooks. That way you have all your servers, networks, images and others in one place, ready for maintenance and management. Since you have it in your deployment tool, all next changes can come via Git and keeping IaaC principles. By running ansible playbooks you can add, remove or change your cloud config.

Keeping current cloud configuration

There are various use-cases of testing on clouds and sometimes it's required to keep the most important resources untouched. Dumped configuration of desired state and all mandatory resources will help to ensure you have all you need before starting the next testing tasks. Just run Transible and get you config dumped in Ansible playbooks. You can edit them and leave there only important things. Executing these playbooks before every cloud work will restore any lacking resources.
Any time you want to "snapshot" your cloud in the current state, just run Transible and using generated playbooks you can return this config later.

Moving current infrastructure to another tenant, cloud, provider, whatever

For example, you need to move your current infrastructure to another cloud in a different hosting provider. In case you don’t have everything in Git or some un-tracked changes were made and you don’t want to lose them. Converting the current cloud to Ansible will freeze cloud config and you can run them with different cloud credentials to deploy everything in a new place.

Release vendor lock

In the next Transible versions it will be possible to convert one cloud configuration to another, for example, Openstack to AWS, or Azure to GCP. This will allow us to prevent hard vendor-lock and help Cloud Ops to maintain and move their infrastructure in more convenient ways.

How it works

Currently, it supports the Openstack cloud only, while others are under development. Using openstacksdk, it requests information from the cloud about current resources. After that using templates of Ansible Openstack modules it creates the playbooks configuration.

Although what if we have hundreds of resources or even more? Having 300 server configurations in a playbook that can end up with 10 kLOCs definitely won't help to maintenance and simplicity. For that Transible knows to optimize cloud resources using the loop cycles of Ansible.
All required data can be separated from playbooks and saved in vars/ folder where can be managed and changed separately, without touching the actual playbooks.
For example images playbook for Openstack can look like:

---
- os_image:
    name: '{{ name | default(omit) }}'
    owner: '{{ owner | default(omit) }}'
    filename: '{{ filename | default(omit) }}'
    min_disk: '{{ min_disk | default(omit) }}'
    disk_format: '{{ disk_format | default(omit) }}'
    properties: '{{ properties | default(omit) }}'
    cloud: my-cloud
    checksum: '{{ checksum | default(omit) }}'
    min_ram: '{{ min_ram | default(omit) }}'
    is_public: '{{ is_public | default(omit) }}'
    state: present
  loop: '{{ images }}'

While images config in vars/ folder includes actual images:

images:
  - name: new_image
    owner: ....
    min_disk: 20
    is_public: false
    filename: /v2/images/...hash.../file
  - name: cloud-image
    checksum: ....
    owner: ....
    min_disk: 8
    is_public: false
    filename: /v2/images/...hash.../file

Parameters that are not defined in variables config will be omitted. So that we'll have the same playbook if weren't optimizing it. But anyway vars optimizations is configurable for any resource type separately in a plugin configuration file:

# Variables optimization configuration
VARS_OPT_NETWORKS = False
VARS_OPT_SUBNETS = True
...

There various plugin configurations that will differ for different user scenarios like management of existing cloud, moving to other providers or deployment from scratch. Some of them can't be "guessed" from cloud configuration which doesn't allow to know what was the initial configuration for the deployment. You can tweak and tune these parameters according to your needs.
Let's take an example of moving to a different Openstack cloud or provider.

Use-case when moving to a new cloud or recreating from scratch

In case you want to recreate your boot volumes and boot servers from them, you will likely configure:

CREATE_NEW_BOOT_VOLUMES = True
USE_SERVER_IMAGES = False
SKIP_UNNAMED_VOLUMES = False
USE_EXISTING_BOOT_VOLUMES = False

This will recreate all required boot and non-boot volumes on a new cloud.
Most likely in the new cloud you'll have new IP pools from your provider, in this case you can't keep the same floating IPs you had before:

FIP_AUTO = True

But in case you'll have the same floating IP range and would like to keep all floating IPs as they are:

FIP_AUTO = False

For DHCP in subnets you can allow the server to have any IP they get from DHCP server:

NETWORK_AUTO = True

But if you want to keep the same IPs exactly in networks, configure:

NETWORK_AUTO = False

Use-case when moving to a new cloud or recreating from scratch

In case you want to rerun your playbook on the existing environment and continue to use it, most likely you don't want to recreate anything. Then your configuration will look like that:

CREATE_NEW_BOOT_VOLUMES = False
SKIP_UNNAMED_VOLUMES = True
USE_EXISTING_BOOT_VOLUMES = True
NETWORK_AUTO = False
FIP_AUTO = True

That way the playbook running will end not changing anything and keeping idempotency. (Be aware that not all Openstack modules are idempotent or support --check run.)
In network configuration you'll have the same IPs in internal networks and subnets are they are today. Although we set FIP_AUTO = True it doesn't mean that the server will get a different floating IP because they already have one, so it will change nothing for them.

Road map

1. Write tests!
2. Create playbooks for user roles, user groups, other openstack services.
3. Create a plugin for Kubernetes.
4. Create a plugin for AWS
5. Create a translator from Openstack to AWS and from AWS to Openstack.

More thoughts and ideas? Please post in comments on issues of the Transible project.