#5 Baking AMIs with Aminator

There are many ways to build AMIs, and this episode will show you how to use the same process Netflix uses by using Aminator. We will then take it one step further, and build AMIs using the Ansible Provisioner as well.

Show Notes

Aminator is a command line tool written in Python by Netflix to make build AMIs easy. Out of the box, Aminator supports Debian and Redhat based OSes by building AMIs using APT or YUM.

To install Aminator on a clean Ubuntu instance:

sudo apt-get install git, python-pip
git clone https://github.com/Netflix/aminator.git
cd aminator/
sudo python setup.py install

This will get you the basics, but by using the CloudFormation template and pre-baked AMI, this is not necessary. The easiest way to get Aminator up and running is by creating a new CloudFormation stack using this template:

https://github.com/Answers4AWS/netflixoss-ansible/blob/master/cloudformation/aminator.json

The template brings up Aminator inside an AutoScaling Group. This means I could not include a simple entry on the Outputs tab of the CloudFormation stack. So you will need go to the EC2 page and find the Aminator instance (tag Name=Aminator) and SSH into that:

$ ansible-ec2 ssh --name Aminator -u ubuntu

To build an AMI, first you will need access to the snapshot of a base AMI. Not many AMI creators (Ubuntu, Amazon, etc) give the public access to the snapshots their official AMIs point to so to get you up and running quickly, here is a list of Ubuntu 12.04 LTS foundation AMIs you can use:

https://github.com/Answers4AWS/netflixoss-ansible/wiki/Foundation-AMIs-for-Aminator

First example, build an AMI with Apache installed. We will do this in us-west-1:

$ sudo aminate -e ec2_apt_linux -B ami-86c0f6c3 apache2

This will build an AMI with Apache installed, with all of the default configuration. You can then launch an instance from the new AMI, go to the public DNS name of that instance and see the lovable default Apache page saying "It works!"

Using APT and YUM is great, but this means whatever you want to install and configure on the OS needs to be wrapped in a package and potentially in a package repository somewhere. By using Ansible, you do not need to build and upload packages into a repository, and can use an Ansible playbook instead. The CloudFormation template already has the Ansible Provisioner for Aminator installed, but if you are not using that, you can install it by running:

$ sudo aminator-plugin install ansible

Aminator also has plugins for Chef Solo and Eucalyptus.

You tell Aminator what combination of plugins you want to use by writing an environment in /etc/aminator/environments.yml. For Ansible, you might use something like this:

ec2_ansible_linux:
    cloud: ec2
    distro: debian
    provisioner: ansible
    volume: linux
    blockdevice: linux
    finalizer: tagging_ebs

The Ansible Provisioner also has a configuration located at /etc/aminator/plugins/aminator.plugins.provisioner.ansible.yml with this content:

enabled: true

# Location and content of local inventory file
inventory_file_path: /etc/ansible
inventory_file: local
inventory_file_content: |
  127.0.0.1

# This is the path to all Ansible playbooks on the Aminator server
# (outside the chroot environment). These will be copied to 'playbooks_path_dest'
playbooks_path_source: /usr/local/netflixoss-ansible/playbooks

# The location to store playbooks on the AMI
playbooks_path_dest: /var/lib/ansible/playbooks

# Set to False to delete all files in 'playbooks_path_dest' before snapshotting
# the volume
keep_playbooks: True

To use your own Ansible playbooks, copy them to the Aminator instance, and change the value of playbooks_path_source. For the NetflixOSS Ansible Playbooks, creating an Asgard AMI would be done by running:

$ sudo aminate -e ec2_ansible_linux -B ami-86c0f6c3 asgard-ubuntu.yml --debug

By adding --debug, you will see the output of the ansible-playbook command.

The Ansible Provisioner also sets one extra variable: ami=True. You can use this variable to have conditional execution in your playbooks. For example, if your playbook installed any services or daemons, you will need to stop them so that the EBS volume can be unmounted successfully. You can use add the following to a Ansible role in your vars/main.yml file:

---
ami_build: ami is defined and ami 
not_ami_build: ami is not defined or not ami

And then use the variables to stop services when the building an AMI, or start them when running the same playbook on a running EC2 instance:

- name: Starting SSH service
  service: name={{ ssh_service_name }} state=started
  when: not_ami_build

- name: Stopping SSH service
  service: name={{ ssh_service_name }} state=stopped
  when: ami_build

When using Aminator, it is also useful to know about Foundation AMIs and Base AMIs. A Foundation AMI is a copy of an OS AMI where your account has access to the EBS snapshots the AMI points to. For example, the Ubuntu Cloud AMIs are public, but the snapshots they point to are not (Ubuntu discussion), which prevents them from being used as Base AMIs by Aminator.

A Base AMI can be either a Foundation AMI, or customized AMI you have built. The Base AMI Netflix use already has Oracle Java, Tomcat, and a few other services already installed and configured. This is an optimization that reduced the amount of time needed to build AMIs.

Resources

comments powered by Disqus