Writings

Technology, open source, personal essays, and everything that isn't climate.

OpenSim Infrastructure Updates: fresh os, git mirror, and automated release building

Yesterday I upgraded the opensimulator.org machine (kindly provided by Adam Frisby) to the latest version of Debian.  The upgrade went seemlessly.  Now that we are on Debian 5.0 we've got some fresher software on the machine to make it possible to provide a few new things as part of the basic OpenSim infrastructure.

OpenSim via Git

We are now mirroring the experimental upstream code (aka subversion trunk) via git.  At least 5 of us on the OpenSim core team have been using git personally with the git-svn bridge for our own OpenSim work (I started doing this nearly a year ago).  Git provides some advantages in making it easy to try things out in a local tree, and throw away branches if things go wrong.  If you read my blog, you know, I love git. :)

While subversion remains our main tree, this git mirror will make it easy for developers (or budding developers) to experiment with this alternative source system.  You can use viewgit to see the git mirror, or clone this via:

git clone http://opensimulator.org/git/opensim

In addition, the viewgit system provides a very handy rss feed for changes, which is another way you can keep up to tabs on what's changing in trunk.  There is an up to 10 minute lag in changes getting into the git mirror from svn, but hopefully that won't bother anyone.

Automated Release Building for OpenSim

Something else I threw together last night was an automated release builder for OpenSim.  One of the challenges we had was getting all the parts of the release sorted out once a release tag was made was sometimes onerous, and meant that a release might only be an subversion tag for days or even weeks before source tarballs of that saw their way into the world.

I've now got a system in place that looks for all numeric tags in our source tree, checks them out, runs prebuild on them, and bundles them up as both a .zip and a .tar.gz.  This means they should be ready to compile with nant or MSVS.  This is running hourly on the OpenSim machine, and publishing all results to http://dist.opensimulator.org.  One of the immediate things you'll see is that it now gives us a full set of historically populated releases.

I'm hoping you enjoy these extra bits of infrastructure for the project.  Please feel free to drop me a comment here if you have any thoughts or questions on them, feedback is always appreciated.

Hey Folks, Please Get a Sense of Humor

There has been some interesting and amazing "outrage" by a bunch of people today of the OpenSim April Fools joke, which was:

  • non destructive change of avatar appearance (no data on disk changed)
  • temporary
  • never put into any release version, or stable tag
  • only in the unstable upstream raw subversion tree

Lots of people got bent out of shape over it, many throwing temper tantrums on the opensim lists.

I'm very glad that so many people feel so entitled that they should be able to run unstable code that the core team says "never run this in production, ever, only run it if you want to help test", not follow any dev discussions, the commit list, or irc (where many people were happily point to the patch to change this), and if anything is out of the ordinary someone owes them.  But that's really not the case.

This is an open source project that a lot of people put a lot of love into, and it's also an open source project in a technology space that is defined in most people's minds by flying purple penises.  This violent reaction to a small amount of fun in trunk is just crazy.  I do hope that everyone who is outraged wrote nasty letters to youtube, google, cnet, and the rest of the internet today.

I we had changed this in a release version, I would consider that over the line.  But trunk is fair game for all manner of experimentation, on April 1 or any other day of the week.  Sometimes fair warning is given, sometimes it isn't.  I was personally responsible for a 5 hour outage on osgrid because the Asset migration that I didn't think would be an issue, and I've broken trunk my fair number of times.

Before you get self-righteous about "you people that have, for free, provided a viable, commericially friendly 3D engine, decided to do something in your unstable tree that doesn't hurt my data", think about the fact that people are doing this on their spare time, and for the love of it.  The net effect of all this grumbling isn't changing whether or not there is an April 1 next year, it's mostly changing whether or not people feel there is a fun loving community here that they still want to support.  Fortunately there has been more user support for being fun loving than against, so the whiners aren't ruining it for everyone, at least not completely.

It really reminds me of a great picture that Roo Reynolds took at the Rails Conf a while back:

Now really, "why so serious?". :)

Who knew that timezone history could be so compelling

I finally decided to find the base zoneinfo files that all timezone data in computing is computed from.  It turns out that the uncompiled files have an incredible amount of history embedded in them, including a number of really interesting stories.  Here are some exceprts:

From Paul Eggert (2001-05-30):

Howse writes that Alaska switched from the Julian to the Gregorian calendar,

and from east-of-GMT to west-of-GMT days, when the US bought it from Russia.

This was on 1867-10-18, a Friday; the previous day was 1867-10-06 Julian,

also a Friday.  Include only the time zone part of this transition,

ignoring the switch from Julian to Gregorian, since we can't represent

the Julian calendar.

...

Since 1970, most of Indiana has been like America/Indiana/Indianapolis,

with the following exceptions:

- Gibson, Jasper, Lake, LaPorte, Newton, Porter, Posey, Spencer,

#   Vandenburgh, and Warrick counties have been like America/Chicago.

- Dearborn and Ohio counties have been like America/New_York.

- Clark, Floyd, and Harrison counties have been like

#   America/Kentucky/Louisville.

- Crawford, Daviess, Dubois, Knox, Martin, Perry, Pike, Pulaski, Starke,

#   and Switzerland counties have their own time zone histories as noted below.

Shanks partitioned Indiana into 345 regions, each with its own time history,

and wrote ``Even newspaper reports present contradictory information.''

Those Hoosiers!  Such a flighty and changeable people!

Fortunately, most of the complexity occurred before our cutoff date of 1970.

Other than Indianapolis, the Indiana place names are so nondescript

that they would be ambiguous if we left them at the `America' level.

So we reluctantly put them all in a subdirectory `America/Indiana'.

...

Shanks writes that Michigan started using standard time on 1885-09-18,

but Howse writes (pp 124-125, referring to Popular Astronomy, 1901-01)

that Detroit kept

#    local time until 1900 when the City Council decreed that clocks should #    be put back twenty-eight minutes to Central Standard Time.  Half the #    city obeyed, half refused.  After considerable debate, the decision #    was rescinded and the city reverted to Sun time.  A derisive offer to #    erect a sundial in front of the city hall was referred to the #    Committee on Sewers.  Then, in 1905, Central time was adopted #    by city vote.

This story is too entertaining to be false, so go with Howse over Shanks.

What's also kind of interesting is the time offsets prior to standardization

Zone    NAME        GMTOFF    RULES    FORMAT    [UNTIL]

Zone America/New_York    -4:56:02 -    LMT    1883 Nov 18 12:03:58 Zone America/Chicago    -5:50:36 -    LMT    1883 Nov 18 12:09:24 Zone America/Los_Angeles -7:52:58 -    LMT    1883 Nov 18 12:07:02

This has now inspired me to request Seize the Daylight from our local library.

Thinking about Debt

I've been thinking about a lot of things in terms of debt recently, and the world looks a bit different if you do that. Debt is borrowing against the future, be that in time, money, energy, health, etc. Debt is what you get when you take short cuts, as you are borrowing from the future.

When your debt is money, it's somewhat easy to understand. You take money from the future you which you have to pay back at some point. It's a little harder to understand in areas that aren't money.

If you create a new piece of code you are creating both value and debt. Debt is created by taking shortcuts, as the software will need to be reworked to reasonably extend it in the future. You take a short cut now to pay for it later, with interest. Every future feature will take longer until you pay back your debt. Refactoring is really all about paying down debt in a responsible way in software.

Most of the time the right approach is to pay off your debt. The other option is bankruptcy (which we are seeing a lot of this week in the financial world). Software bankruptcy is throwing the whole thing out and starting from scratch.

When I started thinking about software development in terms of debt in the last few weeks, lots of things started to make a lot more sense. Shortcuts are debt. Inconsistent interfaces are debt. Inconsistent coding style is debt. Bad or wrong abstractions are debt. Missing documentation is debt. Confusing APIs are debt. If you want a project to move forward more productively you need to eliminate some of your debt, as it's what slows people down (green field code is easy, brownfield is hard).

I'd love to hear about other concepts of debt, and what debt looks like in other media besides software. Please post comments if you are so inclined.

The Schickele Mix Copyright Dilema

One of the constant tensions that exist is the new media age is between preservation of culture and copyrights. Personally this doesn't get summed up any better for me than the fact that Schickele Mix is now lost to us.

Peter Schickele produced 175 episodes of a radio show that explored concepts in music in a very accessible way. I heard it by accident on our local NPR station 7 years ago, and fell in love with it. This was already during one of it's many encores, as new shows had stopped being produced the last 90s. Even though I possess no real musical talent (or perhaps because of that), the show was facinating, and taught me incredible amounts about music. I only wished it was still running somewhere.

Because the show was about music, it played full length songs. The royalty rates for those on broadcast radio were something that was payable at the time, but those rates are substantially higher for online distribution. Hence, there are no archives, and a big piece of culture, one that could get people really excited about music, is now unpublishable due to copyright.

When I was in college, I was always fascinated by the fact that all that still remained of Ancient Greek Theater were 40 some odd plays. How could culture like that get lost? In a digital age it seems incredible that it would be possible to loose important parts of our culture.

Adding stock scripts to OpenSim

Mo Hax and I have started a weekly effort to gather up some of the IBM internally created OpenSim / Second Life content and contribute it to OpenSim as stock content. As OpenSim approaches the 0.6 release, it would be good to have some more reasonable stock content included for those people that aren't Second Life Heads, and have huge inventories of their own stuff sitting on their disks. The results of week one was a single handshake animation, and a lot of understanding on how our stock content system works, and how it should be changed to make it easier to contribute to it.

Yesterday, in between builds and meetings, I decided to refactor a few LSL scripts I had that used unique OSL functions that let you dynamically create textures on objects, both from text drawing commands, and from images of the internet. Those are all now in the OpenSim Library, and accessible for anyone in world. They are under the same license as OpenSim, so do what you will with them. :) (Note: I've found the client caches the inventory trees, so you'll need to clear cache before they show up.)

The scripts contributed yesterday are as follows:

  • osTextBoard - a text board I wrote to do agendas or note taking in world. Modify the script, hit save, and you get the content in your text board texture. Multiple font sizes, colors, and names are used.
  • osWeatherMap - a 3 panel cycling weather map for US weather. This is inspired by the work nebadon did on osgrid.
  • GrafittiBoard - Justin Casey's GrafittiBoard (as seen on osgrid), which is similar to text board, but has an llListen hook so that if you talk on channel 43 it displays it on the board.

Consider all of these as launch points to more complex things. But, they'll at least give people a flavor of what is possible. And you'll get it with every opensim build.

Weekend Rails Hacking

For the past 4 years I've been using evite to manage the RSVPs for our memorial day weekend party. Given that it's a pretty large scale pot luck event, it's helpful to have a system where people can respond with a message that others can see. The reduces my need to field "what should we bring" questions, as you can easily see what everyone else is bringing and react accordingly.

Evite sucks. While it doesn't force attendees to make accounts, it makes it look like it does. The evite.com emails tend to catch as spam. And the interface is now dubious under firefox. The idea is still good, but it hasn't really ramped with the trends in the rest of the service web application space.

One of the key things I wanted in an evite replacement is getting rid of user logins. Given an event, and an email address, you can come up with a unique key that qualifies that person for that event. That means the user just follows a link, and they are in. Links are unique for people. If you make the key a hash of the person's email and some secret seed key for the event, you've got something cryptographically strong as well. No one can modify another person's entry because the key needs to match before you get any info.

Saturday was a rainy day, so I built this system. By Saturday night I had most of it working, and had rolled this out live by Sunday afternoon. This was my first rails 2.0 app, so I needed to catch up on a few things along the way. Things I learned:

  • Rails 2 creates scaffolds in a slightly new way. That threw me for a bit, as I had already built models for most of my objects before creating scaffolds. The new way (putting attributes on the command line) looks like it is designed to make rails tooling easier.
  • ActionMailer is crazy easy. It even does multipart mixed emails really easily. My mhvlug mailer script for month announcements is going to need to be converted to this at some point.
  • Rails has a word_wrap function in the view context. Of course it does, why did I even doubt that.
  • The google maps API is impressive. I had maps based on event location within 60 minutes of signing up for my Google Maps API key.
  • The f= param on maps.google.com is which fuction to drive. q: location query, d: directions. That took a little bit of reading urls to realize.
  • It's pretty easy to integrate mercurial push to auto restart a rails app if it's running under passenger.
  • If you are running multiple versions of rails applicatoins under passenger, delete all the rails links in vendor/ so that it picks up the right rails environment.
  • arround_filter in rails is really handy to catch generic exceptions and dump people off to an error page that isn't the default rails one.

All in all, I was really happy how this turned out. As soon as I get some free time I'll genericize the bits of the app that I coded just for our event, and get this out on rubyforge. I only wish there was a rails equiv of gems, as I've still found that it isn't entirely clear how to best package a rails application as an easy to download open source component.

Seriously underwhelmed by Adobe AIR on Linux

Having been at a Web 2.0 conference for a chunk of the week, I figured I'd finally download an try Adobe AIR, as they announced a Linux version last week at the Linux Collaboration Summit. I am massively underwhelmed.

First off, only 1 of the top 6 air apps on their site actually run on the Linux build of AIR. That app, an rss reader, which is basically a clone of Liferea, but not noticeably better in any way.

Secondly, transparency doesn't work. I installed the Screenboard app and the Ruler app, both of which we useless as they didn't do alpha channel.

Thirdly, their AIR app installer makes a new Gnome application menu for Air Examples each time it installs an application. So I now have 3 Air Examples folders in my application menu, each with 1 application in them.

Finally, air fonts seem to be hard coded to something smaller than my desktop settings, with no clear way to scale them up. Any application framework that doesn't respect my system font settings is pretty useless to me.

Hopefully a lot of these get fixed in the near future. The only thing that AIR seemed to do was a lot of animated menus, which are pretty, but not actually useful. Perhaps if the alpha layer stuff had worked, I'd be a bit more impressed, but without that all the demo apps seem pretty pointless right now. And the one app that I might actually care about, TwittAir, uses functions the Linux AIR runtime doesn't have yet.

Mono 1.9 install script

Unfortunately no one has made ubuntu packages yet, however here is a script that I built based on Dirk's post to automate mono 1.9 installation onto Ubuntu environments.

#!/bin/sh

# This is needed to pick up our built mono for commands
export PATH=/usr/local/bin:$PATH 

apt-get install build-essential bison gawk
apt-get install libglib2.0-dev
apt-get install libpng12-dev libx11-dev libfontconfig1-dev
apt-get install libfreetype6-dev libjpeg62-dev libtiff4-dev
apt-get install libungif4-dev libexif-dev libcairo2-dev
apt-get install libpango1.0-dev libgtk2.0-dev libglade2-dev
apt-get install libgnome2-dev libgnomecanvas2-dev libgnomeui-dev
apt-get install libgnomeprint2.2-dev libgnomeprintui2.2-dev
apt-get install libpanel-applet2-dev libgtksourceview-dev
apt-get install libgtkhtml3.14-dev

BUILDDIR=~/mono-build
mkdir -p $BUILDDIR
cd $BUILDDIR

wget http://go-mono.com/sources/libgdiplus/libgdiplus-1.9.tar.bz2
tar xvf libgdiplus-1.9.tar.bz2
cd libgdiplus-1.9
./configure --prefix=/usr/local
make
make install
cd ..

wget http://go-mono.com/sources/mono/mono-1.9.1.tar.bz2
tar xvf mono-1.9.1.tar.bz2
cd mono-1.9.1
./configure --prefix=/usr/local
make
make install
cd ..

wget http://switch.dl.sourceforge.net/sourceforge/nant/nant-0.86-beta1-src.tar.gz
tar xvf nant-0.86-beta1-src.tar.gz
cd nant-0.86-beta1
make install --prefix=/usr/local
cd ..

wget http://go-mono.com/sources/gtk-sharp210/gtk-sharp-2.10.4.tar.bz2
tar xvf gtk-sharp-2.10.4.tar.bz2
cd gtk-sharp-2.10.4
./configure --prefix=/usr/local
make
make install
cd ..

wget http://go-mono.com/sources/gnome-sharp2/gnome-sharp-2.16.1.tar.gz
tar xvf gnome-sharp-2.16.1.tar.gz
cd gnome-sharp-2.16.1
./configure --prefix=/usr/local
make
make install
cd ..

wget http://go-mono.com/sources/gtksourceview-sharp2/gtksourceview-sharp-2.0-0.12.tar.bz2
tar xvf gtksourceview-sharp-2.0-0.12.tar.bz2
cd gtksourceview-sharp-2.0-0.12
./configure --prefix=/usr/local
make
make install
cd ..

cd mono-1.9
wget  http://go-mono.com/sources/monodoc/monodoc-1.9.zip
unzip monodoc-1.9.zip
cd monodoc-1.9
./configure --prefix=/usr/local
make
make install
cd ../..

wget http://go-mono.com/sources/mono-tools/mono-tools-1.9.tar.bz2
tar xvf mono-tools-1.9.tar.bz2
cd mono-tools-1.9
./configure --prefix=/usr/local
make
make install
cd ..

wget http://ftp.novell.com/pub/mono/sources/mono-debugger/mono-debugger-0.60.tar.bz2
tar xvf mono-debugger-0.60.tar.bz2
cd mono-debugger-0.60
./configure --prefix=/usr/local
make
make install
cd ..

wget http://ftp.novell.com/pub/mono/sources/heap-buddy/heap-buddy-0.2.tar.gz
tar xvf heap-buddy-0.2.tar.gz
cd heap-buddy-0.2
./configure --prefix=/usr/local
make
make install
cd ..

wget http://ftp.novell.com/pub/mono/sources/mono-addins/mono-addins-0.3.1.tar.bz2
tar xvf mono-addins-0.3.1.tar.bz2
cd mono-addins-0.3.1
./configure --prefix=/usr/local
make
make install
cd ..

wget http://ftp.novell.com/pub/mono/sources/monodevelop/monodevelop-1.0.tar.bz2
tar xvf monodevelop-1.0.tar.bz2
cd monodevelop-1.0
./configure --prefix=/usr/local
make
make install
cd ..