[[PageOutline(2-5,Contents,pullout)]] = Orchestrates AWS cloud resources using boto and pychef = == '''WARNING! This plugin is still in alpha.''' == I recommend you ''not'' use this plugin yet on production instances. Although you likely won't lose ec2 instances or chef data, it's still quite immature. It would be best to try out on dev instances to help test the plugin and generate ideas for new features. I'm actively putting it through its paces now at [http://www.juju.com Juju]. - robguttman, 2011 Feb 4 == Description == This plugin is meant to fill the coordination gap between AWS and Chef, especially: * Launch an ec2 instance and apply one or more chef roles * Terminate ec2 instances and delete its chef node (and client) [[Image(instance-grid.png)]] There are additional features and quite possibly more to come, but my intention is to keep the scope down to what's needed to orchestrate between AWS and Chef (and not re-implement either the AWS Management Console or the Chef Server webui). This plugin is highly configurable but is opinionated in some ways. Some of these opinions could be relaxed ([http://trac-hacks.org/newticket?component=CloudPlugin&owner=robguttman submit a ticket!]) but they currently are: * Uses [http://wiki.opscode.com/display/chef/Knife+Bootstrap knife bootstrap] to bootstrap instances - and so your instances' OS must support this method. * Assumes the same credentials and security (e.g., keypair) are used for all instances. * Allows one or more roles (but no recipes) to define a node's run_list. This plugin relies heavily upon [http://boto.cloudhackers.com/ boto] and [https://github.com/coderanger/pychef pychef]. Special thanks to coderanger for his work and responsiveness on pychef. == Configuration == 1. Install the plugin (after downloading and unzipping): {{{ cd cloudplugin/0.12 sudo python setup.py bdist_egg sudo cp dist/TracCloud*.egg /your/trac/location/plugins/ }}} See [http://trac.edgewall.org/wiki/TracPlugins TracPlugins] for more installation details and options. You'll likely need to restart Trac's web server after installation. 2. Enable the plugin in {{{trac.ini}}}: {{{ [components] cloud.* = enabled }}} You can alternatively use the Trac Web Admin GUI to enable any or all rules. 3. Ensure Trac's web server can support long-running requests. Launching an ec2 instance can take several minutes which may be longer than your web server's timeout depending on how you setup Trac. For example, for an apache2 and fcgi setup, you should add the following directive to your Trac's apache conf: {{{ FcgidIOTimeout 900 }}} 4. Configure the main {{{trac.ini}}} section. A minimal configuration would look like this: {{{ [cloud] aws_key = aws_secret = aws_keypair = aws_keypair_pem = /path/to/keypair.pem aws_username = chef_base_path = /path/to/.chef }}} === AWS access configuration === The {{{aws_*}}} options (see example above) are for your AWS credentials and security information. For more info, see [http://docs.amazonwebservices.com/AWSSimpleQueueService/2007-05-01/SQSDeveloperGuide/AWSCredentials.html here] and [http://docs.amazonwebservices.com/AWSSecurityCredentials/1.0/AboutAWSCredentials.html here]. === Chef access configuration === Your Trac server must already have been configured using {{{knife configure}}}. For more info see [http://wiki.opscode.com/display/chef/Bootstrap+Chef+RubyGems+Installation#BootstrapChefRubyGemsInstallation-CreateaKnifeClientforYourLaptop%2FDesktop here]. Set the {{{chef_base_path}}} option to the standard chef directory that contains the {{{knife.rb}}} and client pem files needed to connect with the chef server. Common settings for this option are {{{/etc/chef}}} or {{{/home//.chef}}}. Be sure that the {{{validation.pem}}} file is located where the {{{knife.rb}}} file says it should be. == Bugs/Feature Requests == Existing bugs and feature requests for CloudPlugin are [report:9?COMPONENT=CloudPlugin here]. If you have any issues, create a [http://trac-hacks.org/newticket?component=CloudPlugin&owner=robguttman new ticket]. == Download == Download the zipped source from [download:cloudplugin here]. == Source == You can check out CloudPlugin from [http://trac-hacks.org/svn/cloudplugin here] using Subversion, or [source:cloudplugin browse the source] with Trac. == Examples == This plugin is designed to support multiple AWS resource types, the most notable being ec2 instances. === Instances === The configuration for the ec2 instance resource type has reasonable defaults all of which can be overridden in its own {{{trac.ini}}} section: {{{ [cloud.instance] ... }}} The different configuration options are broken out below. ==== Labels and such ==== {{{ [cloud.instance] title = Instances order = 1 label = Instance description = AWS EC2 instances. }}} The {{{title}}} and {{{order}}} options are used for contextual navigation - see screenshot above. The {{{order}}} should start with {{{1}}}. If {{{order}}} is set to an empty string then the resource will not be shown in the contextual menu (but is still accessible by its url). The {{{label}}} value is displayed in various buttons and forms for the name of a single resource item. The {{{description}}} option is displayed in the grid view - see screenshot above - much like a Trac report description. ==== Field Definitions ==== The fields shown in each view can be configured. Fields are first defined much like Trac custom fields but prefixed with {{{field.}}}: {{{ [cloud.instance] .. field.name = text field.name.label = Name field.name.handler = NameHandler field.run_list = multiselect field.run_list.label = Roles field.run_list.databag = roles # special token field.run_list.handler = RunListHandler field.ec2.placement_availability_zone = select field.ec2.placement_availability_zone.label = Availability Zone field.ec2.placement_availability_zone.options = No preference|us-east-1a|us-east-1b|us-east-1c|us-east-1d .. many more fields .. }}} Each {{{field.*}}} option must map exactly to its chef [http://wiki.opscode.com/display/chef/Attributes attribute] name. For example, {{{ec2.placement_availability_zone}}} above maps to the {{{ec2}}} attribute namespace and the {{{placement_availability_zone}}} attribute within that namespace in chef. It's defined as a {{{select}}} field with a display label and several options separated by a pipe ({{{|}}}) just like Trac custom fields. There are some important differences to Trac custom fields, however: * A field can define a data bag instead of options * A field can define a "field handler" ===== Data Bags for options ===== You may prefer to define a select (or multiselect) field's options by using a chef [http://wiki.opscode.com/display/chef/Data+Bags data bag] instead of listing them in the trac.ini file. For example, a field defined like this: {{{ [cloud.instance] .. field.ec2.ami_id = select field.ec2.ami_id.label = Image ID field.ec2.ami_id.databag = ec2_ami_id }}} will query chef for and use the {{{ec2_ami_id}}} data bag items. The individual data bag items should be formatted like this: {{{ { "id": "ami-xxxxxxxx", "value": "ami-xxxxxxxx", "name": "ami-xxxxxxxx (64-bit)", "order": 1 } }}} The {{{name}}} will be used as the option's displayed name and the {{{value}}} will be used as the option's value. If an {{{order}}} field is provided, it will be used to order the options accordingly (much like custom field ordering). Specifying {{{roles}}} as a data bag has a special meaning (see the {{{run_list}}} field example above) - chef will be queried for the list of [http://wiki.opscode.com/display/chef/Roles roles] instead of a data bag. ===== Field handlers ===== A field handler converts a raw chef attribute value into something more human-readable and vice-versa. For example, the {{{RunListHandler}}} defined above for the {{{run_list}}} field extracts roles from a chef node's run_list and displays them in an easy-to-read way. It also converts data the other direction - i.e., it converts the list of roles from a web form into the correct format to set the node's {{{run_list}}}. Other field handler examples include displaying a url to a web site at a specific port, converting epoch's to dates and times, etc. See [browser:cloudplugin/0.12/cloud/handlers.py handlers.py] for the full list of provided field handlers and [browser:cloudplugin/0.12/cloud/defaults.py defaults.py] for examples of which fields they're used for by default ==== Views ==== In addition to field definitions, you can define which fields should be viewed in which views, their order, and whether or not a field should be read-only in that view. Example: {{{ [cloud.instance] .. crud_view = name, ec2.instance_type, ohai_time, .. crud_new = roles, created_by, created_at, .. crud_edit = roles, created_by, created_at*, .. grid_columns = name, ec2.instance_type, ohai_time, .. grid_sort = ohai_time grid_asc = 0 }}} The {{{crud_view}}}, {{{crud_new}}}, {{{crud_edit}}}, and {{{grid_columns}}} options are lists of field names in the order to be displayed in their respective views. ("crud" stands for "create, read, update, delete".) An asterisk ({{{*}}}) appended to a field name (only applicable for {{{crud_new}}} and {{{crud_edit}}}) indicates that the field should be read-only in that view. All fields in the {{{crud_new}}} and {{{crud_edit}}} views will be used to set or update the chef resource item. [[Image(instance-new.png)]] The {{{grid_sort}}} and {{{grid_asc}}} are for default sorting in the grid view. == Recent Changes == [[ChangeLog(cloudplugin, 3)]] == Author/Contributors == '''Author:''' [wiki:robguttman] [[BR]] '''Maintainer:''' [wiki:robguttman] [[BR]] '''Contributors:'''