The last few weeks I haven’t had much time for Stolen Notebook.
I am one of two programmers at a small startup software company near Madison. I have about 5 hours of free time a day to split up amongst things like eating, working out, reading, and Stolen Notebook. I have even less time if I want to get a reasonable amount of sleep. The last few weeks have been especially hectic since we are nearing a deadline for a second round of investment and need a solid product to show. We have had two rounds of beta testing in order to prepare, which meant extra time spent on the weekends fixing and supporting problems.
Luckily, at Stolen Notebook we have reached a point in development where the workload has started to spread out considerably. The basic engine design is now very solid. Denrei can push source assets through the tools and Tony can use them in the game. Tony has been able to develop game code with very little direct support from me. This means my time can be spent adding new features instead of constant debugging of tools and engine code.
What I am looking forward to is the possibility to further automate our development process.
Some time ago Tony setup buildbot. It automatically builds our engine, tools and documentation when anyone checks in code to subversion. This has freed up a lot of time for everyone. I don’t have to worry about building the latest tools for denrei, they’re automatically available. While this is great for code there has never been a solid process for building games assets.
A long time ago I created a automated game assets build system called SNax (Stolen Notebook Automated eXporter). It was useful…when it worked. It never worked very often. It was very difficult to maintain SNax because the engine was developing so quickly. SNax couldn’t keep up with the changing formats and features. It was eventually left by the wayside.
The next, shortlived, iteration of asset building was to store source assets in subversion. Let me just say this: SUBVERSION IS TERRIBLE FOR STORING LARGE BINARY FILES. Subversion stores two copies of all files it has under revision control. One copy is the one you work on, the other is hidden away in the .svn directory so that you can revert to the original at any point without copying the file from the subversion server. This means that everything takes up twice as much space, and takes twice as much time to checkout. A secondary problem is that diffing huge binary files is very time consuming, so checking in files you’ve changed takes even more time. Denrei tried this system for a short while, and spent all his time waiting upwards of 10 minutes while his work was checked in. The new asset system will have none of these problems.
The major inspiration for this asset building server is this article. The article describes a very simple way to use python to detect file changes in a directory. I’ve done something like this in C before. Doing it in python is much more sane. This script sits and waits for files to be modified in a directory on the server, which is shared on the network, then exports the source asset to the game asset if necessary.
This script fits very neatly into the existing tool set. Let’s consider snscene, the Maya scene exporter. Right now denrei or Tony has to use snscene manually. By associating the Maya file extension (.mb) with the ‘snscene’ command in the directory watching script assets are exported whenever denrei changes a Maya file on the server. The tools don’t have to be modified to support this since the script uses them the same way a user would. The tools can still be used manually and the tools and the automated build system are strictly separated, making maintanence easy.
Enhancing the benefit of this automated build system is out brand new Gigabit network. Denrei tells me that he can transfer files across the network about four times faster than on our old 100 MBit network. I believe him. Gigabit ethernet makes the asset server feasible. It should be almost unnoticible for denrei that he is working on source assets across the network. Combined with the automated asset builder, every time denrei saves everyone should have the latest games assets in a matter of seconds. And no one has to do a thing.