Friday, March 11, 2011

Part 1: Continuous Deployment With Pinax and Jenkins

I admire system admins. They do many things with scripts and commands that are a bit arcane to me. I had heard the term "Continuous Deployment" months ago back when Digg was going through their redesign. Continuous deployment, if you don't know, basically means a script updates code on a live server once a developer commits it. I thought it was an interesting idea, but I'd never be able to pull off such a thing.

Fast forward to today, where we have a Jenkins instance and now multiple developers. Being the lead developer/sys admin on this project by default (I was its sole developer for a while), it was up to me to set up continuous integration and then, if possible, pull off continuous deployment. This blog will describe setting up the CI task.

I had put our code into Hudson months ago, but I had forgotten about it and later found out it wasn't running. It was also having weird connectivity issues, so we figured this would be a great time to upgrade. During these past few months, other people have written much better blogs on how to get Jenkins running with Django/Pinax like Rob Baines. As it turns out, our Jenkins setup is not all that different from the setup described in Rob's blog. I'll lay out the steps and how we diverged from Rob's scripts. Rob's post is a great place to get a little more detail.

  • Jenkins (if you have a Mac, use homebrew and just 'brew install jenkins').
  • virtualenv ('pip install virtualenv')
  • Python 2.4 or higher
  • some kind of database (optional, by default we use SQLite3)

This guide, like Rob's assumes the host system is UNIX (Linux or Mac). Sorry Windows users.

Jenkins plugins:
  • We use Git, so we need the git plugin. You can also install the github plugin if you'd like (provides links to github).
  • Cobertura
  • I don't use the setenv plugin. Rob uses it to set up a path to the virtualenv, but I don't think it's necessary.
Create a Jenkins Job:

Click new job and select "build a free-style software project". Type in a project name and make sure it has no spaces.

Job Settings:
  • Put in a description, link to Github project (if using the Github plugin).
  • In source code management, select Git. Get your project's read-only repo url (no need to do a commit) and specify a branch to build (I don't know what "default" is, so I explicitly put master).
  • If using the Github plugin, you can fill out the repository browser (githubweb) and URL as well.
  • We set Jenkins to poll the repo every 5 minutes, which in cron syntax comes out as "*/5 * * * *"
Build steps:

Step 1: Create virtualenv if it doesn't exist

It's not all that different from Rob's, but since Pinax as of this writing (version 0.7.3) is not available in PyPi, we download the tarball from and install it. This does mean we're somewhat stuck with a certain version of Pinax unless we update it by hand.

Step 2: Install and update dependencies

Similar to Rob's, though I had dumped everything into a single requirements file. In the future, we might want to split up requirements based on whether it's a developer, Jenkins, or the live code. Also, I passed the -q flag to silence pip, otherwise you'd see all these lines in the console that say the requirements are satisfied.

Step 3: Update

Identical to Rob's. We may move to MySQL, but those database settings cannot be in source control. Instead, I'll probably put a different script on the server.

Step 4: Execute Tests

The only thing I changed was that I added some extra parameters to test for nosetests. --with-xunit will create a nosetests.xml for use in reporting test results and --exe tells nosetests not to skip tests that have executable permissions. As for an explanation about the coverage commands, I'll defer to Rob.

Post-build Actions:
  • Check "Publish JUnit report" and give it a path to the nosetests.xml file ("**/nosetests.xml").
  • Check "Publish Cobertura coverage report" and give it a path to the coverage.xml file ("**/coverage.xml").
And that's it for the CI task. So what about continuous deployment?

No comments:

Post a Comment