Tony's Blog

This is an in progress tutorial on how to run buildbot as a Windows XP service. Most of the information comes from these three articles by Grig Gheorghiu:

continuous-integration-with-buildbot
running-buildbot-on-various-platforms
running-python-script-as-windows

At Stolen Notebook we use Buildbot for our automated testing. Buildbot is a system that automates project builds and tests. It helps our team quickly identify errors in our code before they become problems. Buildbot runs after every repository check-in and compiles the code with tests. If the build or tests fail our team receives an email indicating who checked in code last and why the build failed. This aids in catching small problems such as forgetting to add a new file to the repository. A missing file is a very simple problem which can hold up other developers in a big way. Buildbot can also immediately catch larger issues such as tests failing due to new code changes. If you are new to buildbot there is a nice article by Grig Gheorghiu on setting up buildbot at continuous-integration-with-buildbot. If you are setting up buildslaves on multiple platforms he also has a nice article at running-buildbot-on-various-platforms.

Once you get your buildbot farm going you might want to run your Windows NT/XP/2003 buildslaves as windows services. Our WindowsXP buildslave serves multiple purposes. Different users log in and out of it on a regular basis. Running buildbot from the command line doesn’t work very well in this situation. The easiest solution is to run buildbot as a Windows Service. This allows buildbot to start running when Windows begins. Buildbot then runs in the background despite which users log in or out of the machine. Some small issues may arise when turning buildbot into a service so it is best to make sure buildbot is working from the command line first.

Here is an article about running a python script as a windows service running-python-script-as-windows

I created a buildslave user and got my slave working on the command line under that user’s account. I then created a service called BuildSlave and set it to run under the buildslave user. You can do this by going to Control Panel-> Administrative Tools->Services. Right click on the Buildslave service and select Properites. Go to the “Log On” tab and under “Log on as:” select “This account:” Now type in the buildslave username (I named mine buildslave) and password. The service will now run as the buildslave user. Some environmental variables might not get set. See the Problem Solving section below for a more detailed description.

There are different methods for launching the python script. The one mentioned in the article uses a batch file.

UPDATE: A comment by jdpipe mentions a better way to launch buildbot by using a python script instead of a windows batch file. Check out his article at http://ascendwiki.cheme.cmu.edu/BuildBot.

For buildbot you would make a batch file with the contents:

C:\Python24\python C:\buildbot\bin\buildbot start c:\buildslave\game

Make sure to substitute the correct paths for python, buildbot, and the buildslave directory. I named my batch file startslave.bat.

It should be mentioned that if the batch file method is used then stopping the service might not kill the python.exe process. You will have to kill it manually.

Problem Solving:

Buildslave stays in ACTIVE state:
A problem I had was when a buildslave would do a build and then never return to the IDLE state. On the next build the buildmaster would go through the buildslaves looking for IDLE ones but the buildslave would still be in the ACTIVE state and get skipped. Since it would never return to the IDLE state it would never do another build. I believe this problem was caused by not calling a superclass’s __init__ function from a subclass in my master.cfg file. If you have a configuration where you are subclassing a buildbot class(Such as subclassing step.ShellCommand to change the name of a build step) make sure you call the superclass’s __init__ function from the subclass’s __init__ function. If you don’t the effects could be small and unnoticeable at first but cause the buildslaves or buildmaster to behave in very strange ways because certain classes were not initialized properly.

Buildslave hangs or doesn’t start a build when running as a service:
When buildbot is started as a service the windows environment may be different even if you run the service as the same user you ran buildbot as from the command line. Certain environment variables which are set and valid when you run your slave from the command line might be invalid or not even exists when the slave is run as a service. Two big variables are HOMEPATH and HOMEDRIVE. If your buildbot setup requires some programs which rely on config files in the buildslave user’s home directory they might use those environment variables to find where those files reside. If the variables don’t exist those programs might use default options which cause them to just sit there and do nothing. When buildbot receives a command it dumps the environment just before running the command. A good idea is you run the buildslave from the command line and take a look at the environment. Then run the buildslave as a service and compare the two environments. You can find all logged messages in the twistd.log file which resides in your buildslave’s configuration directory. After comparing environments I found the service environment was missing both the HOMEPATH and HOMEDRIVE variables. My ssh setup required config files in those directories and was unable to find them. I then set those variables in my startslave.bat(which is launched by the BuildSlave service) prior to launching buildbot. It now looks like this:

set HOMEDRIVE = "C:"
set HOMEPATH = "\Documents and Settings\buildslave"
C:\Python24\python C:\buildbot\bin\buildbot start c:\buildslave\game