Upstream Conductor Development

Getting Started: Defining Non-Conductor APIs and Credentials

So you want to get your hands dirty hacking on the latest Conductor code? Great! There is a script available, “bootstrap.sh,” in the Aeolus Incubator dev-tools project that can help you set up a development environment quickly, but before we get to that… you do have an existing Image Factory, Image Warehouse and Deltacloud installation at your disposal, right?

[Hint: To quickly set those up, just run "aeolus install-all" and "aeolus-configure" as described in this blog post. It also sets up its own instance of (non upstream) Conductor via rpm which you can ignore, stop or leave running for comparison purposes.]

As indicated in README.md, there are a few enviornment variables you need to set in your shell which point to the Image Factory, Image Warehouse and Deltacloud API’s, namely $FACTER_DELTACLOUD_URL, $FACTER_IMAGEFACTORY_URL and $FACTER_IWHD_URL. Chances are your url’s will look the same as the url’s in the README.md with the only difference being the hostname. If your Image Factory, Image Warehouse and Deltacloud are all running on localhost on the default ports, you actually don’t need to explicitly define these environment variables (bootstrap.sh will take care of it and point them to localhost).

The other environment variable you need to be aware of before firing off bootstrap.sh is $FACTER_OAUTH_JSON_FILE. This just needs to point to an existing oauth.json file that contains Image Factory and Image Warehouse credendtials, e.g. “/path/to/my/oauth.json”. If you took the shortcut of running “aeolus install-all” and “aeolus configure,” this file is located at /etc/aeolus-conductor/oauth.json. Note by default, bootstrap.sh looks for oauth.json at /etc/aeolus-conductor/oauth.json, so you don’t need to explicitly set $FACTER_OAUTH_JSON_FILE if you already have it there.

Getting Started: Deciding Who, Where and What Ruby

Now you are past the hard part, though hopefully the above section wasn’t too hard (you didn’t skip that did you?). The main things left to decide are:

  • Which system user should we use to check out the Conductor code? Set $DEV_USERNAME to this value (or if you are happy with the default user “test” you need not define it– boostrap.sh will add the system user “test” if necessary in this case).
  • What is the top level directory where you want the conductor, aeolus-cli, aeolus-image-rubygem projects checked out to? Set this to $WORKDIR (or if you are happy with the default /tmp/test, you need not define it). bootstrap.sh will create this directory, if needed.
  • What port do you want Conductor to run on? Set this to $FACTER_CONDUCTOR_PORT (or if you are happy with the default port of 3000, you need not define it).
  • What version of ruby do you want to run (via rbenv)? Set this to $RBENV_VERSION (or if you are happy with using system ruby, do not set it). More information about rbenv can be found in README.md.

Run boostrap.sh

You’ve set the environment variables you need set, so now you are ready to do (as root):

   curl https://raw.github.com/aeolus-incubator/dev-tools/master/bootstrap.sh | /bin/sh -x

This may take awhile to complete as it is checking out a few repositories and is going to “bundle install” quite a few ruby dependencies (locally to the conductor checkout, not to your system ruby libs). Keep your eye on the output. You shouldn’t see any errors… well except for this one annoying puppet-related error:

 Error: Could not prefetch package provider 'yum': The yum provider can only be used as root

but you can safely ignore that.

bootstrap.sh will install any system-level dependencies it needs to complete (e.g., puppet), though there should be relatively few of them. bootstrap.sh is known to work on Fedora 16, Fedora 17 and RHEL 6.

It Ran, Now What?

bootstrap.sh starts three ruby processes (as $DEV_USERNAME). If you do “ps -ef | grep ruby”, you see something like:

test      8590     1  1 16:24 ?        00:00:39 /usr/bin/ruby script/rails server -p 3000
test      8610     1  0 16:24 ?        00:00:05 ruby dbomatic/dbomatic --log log --pid-file tmp -n
test      8615     1  0 16:24 ?        00:00:06 ruby /tmp/test/conductor/src/bundle/ruby/1.8/bin/rake jobs:work

If you point your browser to http://[your-conductor-host]:[your-conductor-port] you should be able to log in to Conductor with admin/password.

Note that all three processes are required for a fully functional Conductor instance. “rake jobs:work” (AKA delayed jobs) is responsible for firing off build requests, for instance. “dbomatic” is responsible for updating the database with information from Image Factory. “rails server” is just Conductor’s webserver, obviously.

If you killed these three ruby processes and wanted to start them back up again, you would execute the shell commands (below) as $DEV_USERNAME, assuming that $WORKDIR is /home/test/w1. Note that if you defined $RBENV_VERSION in bootstrap.sh, you need to make sure rbenv is in your path (see README.md).

  $ cd /home/test/w1/conductor/src
  $ bundle exec "rails s -p 3000"&
  $ bundle exec "rake jobs:work"&
  $ bundle bundle exec "ruby dbomatic/dbomatic --log log --pid-file tmp -n"&

Also, notice that you did not need to explicitly set any enviornment variables as we did in bootstrap.sh (though of course you’ll need to know where your $WORKDIR was to cd into it and probably would want to start rails back up on the same $FACTER_CONDUCTOR_PORT).

Compartmentalization

The philosophy behind boostrap.sh and supporting puppet scripts is to keep the development environment as compartmentalized as possible within the $WORKDIR. Once boostrap.sh runs, the development user $DEV_USERNAME should be able to work in that enviornment and execute typical “bundle exec” rails tasks (thanks of course to Bundler). All Gemfile dependencies are pulled locally under $WORKDIR/conductor/src/bundle. Sqlite is used and lives under $WORKDIR/conductor/src/db.

Thus, it is easy to set up any number of development environments on a single host. Obviously, $FACTER_CONDUCTOR_PORT and $WORKDIR must be unique to each environment.

Bells, Whistles, Pull Requests

With bootstrap.sh, it is extremely easy to set up a one-off environment for either a quick test or some serious development. One neat feature is the ability to automatically apply an active pull request by setting something like FACTER_CONDUCTOR_PULL_REQUEST=117 (the value must be the integer value of the pull request) prior to running bootstrap.sh. Another neat feature is being able to specify $RBENV_VERSION. Curious if all the rspec tests will pass given a certain ruby version? Just set $RBENV_VERSION before you run bootstrap.sh, and the ruby version you specify will be installed locally to $DEV_USERNAME’s home dir (if necessary) along with rbenv itself (if necessary) — no need to set up rbenv on your own first.

I’m a Complete Newb, a Little Help?

Say you’ve run bootstrap.sh with the defaults. Here are the commands you would use start from scratch– dropping the database and setting everything up again. The $WORKDIR is assumed to be /tmp/test. You would run these as the “test” (or $DEV_USERNAME) user, of course.

  $ killall ruby # kill the three ruby processes bootstrap.sh started
  $ # (make sure the processes are dead)
  $ cd /tmp/test/conductor/src
  $ bundle exec "rake db:drop:all" # don't worry if you get informed
  $ #                              # of a production or test db not existing
  $ bundle exec "rake db:migrate"
  $ bundle exec "rake db:setup"
  $ bundle exec "rake dc:create_admin_user"
  $ bundle exec "compass compile"
  $ bundle exec "rails s -p 3000"&
  $ bundle exec "rake jobs:work"&
  $ bundle exec "ruby dbomatic/dbomatic --log log --pid-file tmp -n"&

One thought on “Upstream Conductor Development

  1. Pingback: Bringing Consistency to the UI | Aeolus Project Blog

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>