Err, as per this post:
http://forum.ppcgeeks.com/2067605-post1702.html
I'm going to leave my battery out for a long time, and then try again. I guess the issue is that it was shut down uncleanly and it remembers that it should be on.. so it immediately powers on as soon as it has enough charge, but its not enough to finish the boot.
If it doesn't work by morning, I'll have someone else charge my darn battery.