Version 21 (modified by 13 years ago) (diff) | ,
---|
Contents
Orchestrates AWS cloud resources using boto and pychef
Notice: This plugin is unmaintained and available for adoption.
WARNING! This plugin is in alpha.
I recommend you not use this plugin yet on production instances. Although you likely won't lose ec2 instances or chef data, it's still quite immature. It would be best to try out on dev instances to help test the plugin and generate ideas for new features. I'm actively putting it through its paces now at Juju.
- robguttman, 2011 Jan 28
Description
This plugin is meant to fill the coordination gap between AWS and Chef, especially:
- Launch an ec2 instance and apply one or more chef roles
- Terminate ec2 instances and delete its chef node
There are additional features and quite possibly more to come, but my intention is to keep the scope down to what's needed to orchestrate between AWS and Chef (and not re-implement either the AWS Management Console or the Chef Server webui).
This plugin relies heavily upon boto and pychef. Special thanks to coderanger for his work and responsiveness on pychef.
Configuration
- Install the plugin (after downloading and unzipping):
cd cloudplugin/0.12 sudo python setup.py bdist_egg sudo cp dist/TracCloud*.egg /your/trac/location/plugins/
See TracPlugins for more installation details and options. You'll likely need to restart Trac's web server after installation.
- Enable the plugin in
trac.ini
:[components] cloud.* = enabled
You can alternatively use the Trac Web Admin GUI to enable any or all rules.
- Configure the main
trac.ini
section:[cloud] label = Cloud aws_key = <aws-access-key> aws_secret = <aws-secret-key> aws_keypair = <ec2-key-pair> chef_instancedata_file = /path/to/instance-data.json
The label
option above is the name to be used in the top navigation - see screenshot above.
AWS credentials
The aws_*
options are your AWS credentials and the key pair used by your ec2 instances.
Chef configuration
The chef_instancedata_file
option is the absolute path to the ec2 instance/user data generated by chef's knife command:
% knife ec2 instance data role[<desired-role>] > /path/to/instance-data.json
The plugin will use this file for the instance data replacing the role information as needed. As the file contains the validation.pem
data, be sure to secure it accordingly. In cases where you need different credentials and/or chefserver url, you can use the following option to specify a base directory which must contain the typical .chef
directory and its contents plus (optionally) a validation.pem
file within the .chef
directory:
[cloud] .. chef_base_path = /path/to/dir/that/contains/.chef
It's currently assumed that you're using one of Opscode's AMIs or a homegrown AMI derived from it. These AMIs use the instance's id as the node name. However, you may have recipes which for one reason or another change the node's name after the first chef-client run. If so, you need to specify the boto field that maps to the final node name using this config option:
[cloud] .. boto_field_node_name = private_dns_name
Adding this option will cause the plugin to create a new node of that name upon launching a new ec2 instance and once it checks in will remove the original node. Otherwise you'll have two chef nodes for the same ec2 instance - something you likely don't want. Yeah, this is a bit complicated - an area ripe for improvement.
The plugin is intended to support multiple AWS resource types in addition to instances as needed for orchestration with chef. You specify the default resource to view using this config option:
[cloud] .. default_resource = instance
Each AWS resource type such as instance
above is configured independently. See the examples section below for how to specify these resources.
Bugs/Feature Requests
Existing bugs and feature requests for CloudPlugin are here.
If you have any issues, create a new ticket.
Download
Download the zipped source from [download:cloudplugin here].
Source
You can check out CloudPlugin from here using Subversion, or browse the source with Trac.
Example
Each supported AWS resource is configured independently in a separate section in trac.ini
. (The plugin currently only supports ec2 instances.) Each section must be named as follows:
[cloud.<resource_name>] ...
For example, a section named [cloud.instance]
defines a resource named instance
. The resource name is used in the uri, so for example, this instance
resource would be accessed at:
http://trac.domain.com/cloud/instance
You can name resources whatever you want as long as they're unique.
Instances
Here's how to configure ec2 instance resources. It's best to explain the configuration by example and piecemeal:
[cloud.instance] class = Ec2Instance title = Instances order = 1 label = Instance description = AWS EC2 instances.
class
- must map to the exact python class name for that instance. For ec2 instances, it'sEc2Instance
.title
- used in the sub-navigation (aka contextual) menu - see screenshot above.order
- defines the order in the contextual menu (much like custom field ordering).label
- used in several views to describe a single one of these resources.description
- used in the grid view - see screenshot above.
Now it gets a bit more intense (the following should really all be on one line in trac.ini
):
[cloud.instance] .. fields = name < NameHandler, ec2.instance_id as Instance ID, run_list_ as Roles < RunListHandler, created_by as Created By < AuthorHandler, created_at_ as Created At < EpochHandler, ohai_time_ as Last Check-in < AgoEpochHandler, ec2.instance_type as Instance Type, ec2.hostname as Private Hostname, ec2.public_hostname_ as Public Hostname < SshHandler, ec2.placement_availability_zone as Availability Zone, ec2.ami_id as Image ID, supervisord.ssl_port_ as supervisord < HttpsHandler
Whoa! The fields
option is a list of those chef attributes (aka fields) where you want to provide either a display name, a format handler, or both. So for example, ec2.instance_type as Instance Type
above will use 'Instance Type' wherever the name of the ec2.instance_type
field is to be displayed. This as
notation should be familiar to those who have written Trac report SQL (although no quotes should be used here).
Only slightly more complicated are handlers to format fields. So for example, ohai_time_ as Last Check-in < AgoEpochHandler
has a display name as we've seen plus the < AgoEpochHandler
directs the plugin to use a specific handler to format the ohai_time
field's value. In this example, the AgoEpochHandler
handler converts an epoch to a string format such as "0:08:36 ago" meaning "8 minutes and 36 seconds ago" as shown in the screenshot above. But wait, what's up with the underscore after the ohai_time_
? In most cases when using a handler, you should postfix the field's name with an underscore (_
) so that the re-formatted value doesn't get written back to the field's value. [I'm likely to get rid of the underscore business to make it simpler, but that's what it is for now. - rhg] See the handlers.py
file for a full list of provided field handlers.
grid_index = node grid_columns = name, run_list_, created_by, ohai_time_, ec2.instance_type, ec2.placement_availability_zone grid_sort = created_at_ grid_asc = 0
The grid_*
options configure the grid view of the resource - i.e., the view shown in the screenshot above:
grid_index
- must map to the chef search index name. For ec2 instances, this isnode
.grid_columns
- the list of chef attributes to display in order. The column names will use any display names fromfields
and its cell values will be formatted using any handlers fromfields
.grid_sort
- the default sort attribute/field. You can resort the grid by clicking on the column name.grid_asc
- the default sort direction. You can change the sort direction by clicking on the same column name.
crud_resource = nodes crud_view = name, ec2.instance_id, run_list_, created_by, created_at_, ohai_time_, ec2.instance_type, ec2.hostname, ec2.public_hostname_, ec2.placement_availability_zone, ec2.ami_id, supervisord.ssl_port_ crud_new = run_list_||, created_by, ec2.instance_type|, ec2.ami_id|, ec2.placement_availability_zone|us-east-1a|us-east-1b|us-east-1c|us-east-1d crud_edit = run_list_||, created_by, ec2.instance_type*, ec2.ami_id*, ec2.placement_availability_zone*
The crud_*
options configure aspects for creating, reading updating, and deleting (i.e., CRUD) the resource:
crud_resource
- must map to the chef resource name. For ec2 instances, this isnodes
.crud_view
- the list of chef attributes to display in order. The field names will use any display names fromfields
and its cell values will be formatted using any handlers fromfields
.crud_new
andcrud_edit
- the list of augmented chef attributes to display for editing in order. The field names will use any display names fromfields
and its cell values will be formatted using any handlers fromfields
.
The field names in crud_new
and crud_edit
are augmented to provide additional metadata so the plugin knows how to properly configure the form. So for example:
ec2.placement_availability_zone|us-east-1a|us-east-1b|us-east-1c|us-east-1d
The pipe (|
) after the field name indicates that the form should use a select/dropdown menu for this field. If the select/dropdown should support multiple selections, then use two pipes instead of one as in run_list_||
. The remainder of the definition is a pipe-delimited list of options for this field (much like how custom field options are defined). If there's no list of options, then the plugin will search a chefserver data bag of the same name as the field for the list of options. So for example, ec2.instance_type|
will direct the plugin to search for a data bag on the chefserver named ec2_instance_type
(periods are invalid and so are replaced with underscores). The individual data bag items should be formatted like this:
{ "id": "m1_large", "value": "m1.large", "name": "m1.large - Large (64-bit)", "order": 3 }
The name
will be used as the option's displayed name and the value
will be used as the option's value. If order
fields are provided, then they'll be used to order the options accordingly (much like custom field ordering).
Lastly, if you want to have a field displayed but as read-only, then the field name should be immediately postfixed with an asterisk as in ec2.ami_id*
above. The crud_new
config example above results in the following screenshot:
Recent Changes
- 14004 by rjollos on 2014-07-12 18:27:48
-
Correct
license
keyword argument insetup.py
. Refs #11832.
- 13996 by rjollos on 2014-06-25 00:56:32
-
Fixed author in COPYING file. Refs #11832.
- 13994 by rjollos on 2014-06-25 00:51:36
-
Changed to 3-Clause BSD license with permission of author. Refs #11832.
(more)
Author/Contributors
Author: robguttman
Maintainer: robguttman
Contributors:
Attachments (4)
- nav.png (2.2 KB) - added by 13 years ago.
- instance-new.png (28.5 KB) - added by 13 years ago.
- progress.png (15.6 KB) - added by 13 years ago.
- instance-grid.png (118.5 KB) - added by 13 years ago.
Download all attachments as: .zip