Using Ansible to provision VMs on AWS
I have been asked on several occasions to show how to use Ansible to provision VMs on Amazon Web Services (AWS). This is “commoditization virtualization” on demand just by running a single playbook, which is pretty cool.
Why automation in the first place?
If your reading this article and have any experience configuring A Unix/Linux/Windows server, whether it is a mail server, web server, whatever, you know how time consuming it is to:
- Partition the disk
- Create user accounts
- Install software packages and updates
- Configure the server application
You had to wait for the packages to install, configure and test the application to make sure it runs and that can take a few hours.
Now, that the servers are virtual and live in a cloud machine somewhere and you now have to configure more than a dozen of them… that’s a lot of time and you have better things to do. Tools like Ansible are the answer to configuring multiple machines.
Spining up VMs using AWS web console
AWS allows you to log into a web console, choose your VM image and bring them up one at a time. It will give you ssh credentials to allow you to login to the VMs you just made and from there, use Ansible to manage and configure them. Would it be nice if you could manage the provisioning of VM instances from Ansible? Yes you can…but there is a bit of work to make it work.
I will describe the way you can do it with several step script and (in another article) programmatically.
How do I know how many VMs I have in inventory?
The challenge of dynamic inventory is the program/playbook does not know what is in the inventory ahead of time. However, if we apply the cattle not pets approach and let Ansible take care of itempotents of the VMs (it won’t clobber the VMs or exceed the constraints of number of VMs that exist) then this can make our lives easier.
Without knowing the inventory, and checking it ahead of time, you are running blind.
Programmatically is the best way to manage and track dynamic inventory and use Ansibles modules to provision VMs in AWS.
Using a playbook to provision VMS.
In my opinion this is a clunky way to use Ansible.
The problem with this way is there is no clean way to see what is the current inventory that is on AWS, you have to run a separate program before running the playbook so you can see what is currently in inventory.
When this is done you end up writing three or four separate scripts to manage this process. In the long run, this becomes difficult to maintain since you have to look at other scripts to understand what is going on.
Writing maintainable code is a key principle.
Build the playbook to provision AWS cloud services.
Playbook is set to local host.
AWS keys are needed for AWS account access.
BOTO Python API libraries are installed.
Just in case you are not aware, BOTO is the API AWS uses for programmatically managing AWS services.
AWS cloud account
Log into AWS Management Console
Under user account, select “security credentials”
In the left hand column, select user
Select the security tab.
Look for security access key.
This is what you will need for boto/ansible.
I have created a Vagrant file with an Ansible playbook for managing AWS through a Linux VM created and managed by Vagrant.
First install VirtualBox then install Vagrant.
Download from Github the Vagrant Ansible AWS files.
Change into the vagrant file directory and type:
It will take a while for all the dependencies to be downloaded.
Once vagrant is fully up, type:
to access the shell of the vm.
Preparing for instances.
Change directory to the ansible playbook directory and modify the following files:
and add your AWS keys.
Provisioning AWS instances.
from the shell, type:
to start provisioning instances in AWS.
You can watch from the AWS console the instances being provisioned.
from the shell, type:
to terminate the ec2 instances that were provisioned in your account by the provisioning playbook.
You can watch the instances terminated from the AWS console.
This is only the beginning…
With these examples, we just created self contained machines just by running an Ansible playbook. However, we can setup a virtual container network that allows you to place in a private network such items as private networks where you have access to file servers database servers an “internal” and “external” network with a “firewall”. and more complex designs.
I may cover these examples in future articles.
In the meantime, have a great day.