Using Ansible to provision VMs on AWS

Using Ansible to provision VMs on AWS

I have been asked on several occasions to show how to use Ansible to provision VMs on Amazon Web Services (AWS). This is “commoditization virtualization” on demand just by running a single playbook, which is pretty cool.

Why automation in the first place?

If your reading this article and have any experience configuring A Unix/Linux/Windows server, whether it is a mail server, web server, whatever, you know how time consuming it is to:

  • Partition the disk
  • Create user accounts
  • Install software packages and updates
  • Configure the server application
  • etc…

You had to wait for the packages to install, configure and test the application to make sure it runs and that can take a few hours.

Now, that the servers are virtual and live in a cloud machine somewhere and you now have to configure more than a dozen of them… that’s a lot of time and you have better things to do. Tools like Ansible are the answer to configuring multiple machines.

Spining up VMs using AWS web console

AWS allows you to log into a web console, choose your VM image and bring them up one at a time. It will give you ssh credentials to allow you to login to the VMs you just made and from there, use Ansible to manage and configure them. Would it be nice if you could manage the provisioning of VM instances from Ansible? Yes you can…but there is a bit of work to make it work.

I will describe the way you can do it with several step script and (in another article) programmatically.

How do I know how many VMs I have in inventory?

The challenge of dynamic inventory is the program/playbook does not know what is in the inventory ahead of time. However, if we apply the cattle not pets approach and let Ansible take care of itempotents of the VMs (it won’t clobber the VMs or exceed the constraints of number of VMs that exist) then this can make our lives easier.

Without knowing the inventory, and checking it ahead of time, you are running blind.

Programmatically is the best way to manage and track dynamic inventory and use Ansibles modules to provision VMs in AWS.

Using a playbook to provision VMS.

In my opinion this is a clunky way to use Ansible.

The problem with this way is there is no clean way to see what is the current inventory that is on AWS, you have to run a separate program before running the playbook so you can see what is currently in inventory.

When this is done you end up writing three or four separate scripts to manage this process. In the long run, this becomes difficult to maintain since you have to look at other scripts to understand what is going on.
Writing maintainable code is a key principle.

Build the playbook to provision AWS cloud services.

Playbook is set to local host.

AWS keys are needed for AWS account access.

BOTO Python API libraries are installed.

Just in case you are not aware, BOTO is the API AWS uses for programmatically managing AWS services.

AWS cloud account

Log into AWS Management Console

Under user account, select “security credentials”

In the left hand column, select user

Select the security tab.

Look for security access key.

This is what you will need for boto/ansible.

Running Vagrant

I have created a Vagrant file with an Ansible playbook for managing AWS through a Linux VM created and managed by Vagrant.

First install VirtualBox then install Vagrant.

Download from Github the Vagrant Ansible AWS files.

Change into the vagrant file directory and type:

vagrant up

It will take a while for all the dependencies to be downloaded.

Once vagrant is fully up, type:

vagrant ssh

to access the shell of the vm.

Preparing for instances.

Change directory to the ansible playbook directory and modify the following files:


and add your AWS keys.

Provisioning AWS instances.

from the shell, type:

ansible-playbook AWS-provision.yml

to start provisioning instances in AWS.

You can watch from the AWS console the instances being provisioned.

Terminating instances

from the shell, type:

ansible-playbook AWS-terminate.yml

to terminate the ec2 instances that were provisioned in your account by the provisioning playbook.

You can watch the instances terminated from the AWS console.

This is only the beginning…

With these examples, we just created self contained machines just by running an Ansible playbook. However, we can setup a virtual container network that allows you to place in a private network such items as private networks where you have access to file servers database servers an “internal” and “external” network with a “firewall”. and more complex designs.

I may cover these examples in future articles.

In the meantime, have a great day.

The virtues of Ansible – Running Ansible on Windows

Devops and Ansible

The mantra of DevOps (Development Operations) is Infrastructure is Code.
For those who are not familiar with DevOps, it is a culture and philosophy of computer and network infrastructure management geared around development and delivery of software and services. DevOps uses tools geared towards automation of infrastructure, delivery of software and services.
Ansible is one of those indispensable tools used in DevOps. With Ansible, you can manage your whole network infrastructures , deploy and manage virtual machines, install and configure VMs, provision and update software and services, etc. Written in python, and the configuration files are “programmed” in YAML (Yet Another Markup Language). It is a easy to learn tool for mass provisioning of Linux, Windows, Mac and virtual infrastructure using scriptable command line.
Windows operating system is ubiquitous and is probably one of the easiest, lowest cost hardware/software combinations to get for your testing and development and production environment.

The Windows Dilemma

However if you like me, are running Windows, there is one problem, Ansible does not run on Windows. The developers of Ansible have no intention of porting it to Windows (at this time). Even though, there “is” a way to use Linux like libraries on Windows (using things like CygWin) , however, it’s not very stable. You need a stable platform to do mission critical work. So, how do I run Ansible in Windows?

Introducing Vagrant

The answer to this dilemma is to use Virtualization. “Wait a minute…”, you might say, “Isn’t this a clunky and bulky way to run a tool by getting a clunkier interface to run it on?” This is where a virtualization environment for developers like Vagrant comes into play. Vagrant, from command line allows you to do some cool things:

1 – Use your favorite Windows based editor to edit your playbooks in which you can make changes to your files THEN run them without repackaging the image (as in the case of Docker).
2 – With command line within PowerShell, QUICKLY spin up your Linux VM to run Ansible.
3 – From the same command line, enter the Linux VM command line and run Ansible commands and run your playbooks.
4 – If you want to run the Vagrant VM for a dedicated task involving Ansible, with one command, you can do just that.

For those not familiar with Vagrant, this is a piece of cake.

How to set up Ansible in Vagrant

For the purpose of demonstrating how to do this in Windows, I will show you how to do this in a Windows 7 SP1 machine. I will also assume you are running the latest version of VirtualBox hypervisor. Vagrant will also work with VMWare and HyperV and other versions of Windows (7/8/10), but this will be left (for now) as an exercise you can do on your own.

Also, my preferred way to add software to Windows machines is to use a package manager like Chocolatey. Using automation with package management is the DevOps way. Follow this link for instructions installing Chocolatey.

Steps for setting up Vagrant

1 – Download and install VirtualBox

a – use chocolatey
C:\> choco install virtualbox

b – manually install
Download from here:

2 – Download and install vagrant

a – use chocolatey
C:\> choco install vagrant

b – manually install
Download from here:

3 – Download this vagrant file from the git repository and unpack it. This is a Vagrantfile template. You can use git to get the file.

4 – Change directory to vagrant directory and run “vagrant up”

5 – Access VM shell, run “vagrant ssh” It will take you into a Linux shell.

6 – To exit out of the VM shell, type ‘exit’ and hit enter. You will be back to the powershell command prompt.

7 – To shut down vagrant and the VM, run “vagrant halt” and it will shut down the VM.

I would recommend reading the file in the directory for additional details.

How do these files work?

This is not a full tutorial on how to work with Vagrant, just enough to get you running with Ansible on Windows. We may cover the particulars of Ansible and Vagrant another time.

What is a Vagrant file?

The Vagrantfile tells vagrant how to provision the VM. The VM will be installed with Python and Ansible. It’s a file written in the Ruby programming language. It’s pretty straightforward how you can configure it to work with your development environment. Anyhow, from here, you can experiment with Ansible on the VM itself.

Once you access Ansible through Vagrant, you can start using Ad Hoc commands and writing playbooks for Ansible and executing them right there. Do some searching on the internet on what you can do with Ansible. It’s a very powerful tool.

That was quite a bit of info, how do I wrap my head around this?

That is the unfortunate nature of any profession, especially the technology profession.  You need to know many different tools and concepts in order to use them, and many of the tools by themselves don’t make much sense until you find a context to use these tools.  Each one of the tools and technologies I used in this intro have their own use in the DevOps area.  You have to dive right in and play with them in order to find out how they can help you to use them in your own particular situation.   I may cover each of these tools  individually in a future article.

Until then, have a good day.