I added 3GiB of memory to the server, and bumped the Java process up to using 2 from 1.
So I've increased the player limit to 15. Hopefully it can handle it without much difficulty.
Also, apologies for the recent crashes - between minecraft.net being down and hey0's mod being kinda flakey the last (beta version), there's been some segfaults.