[mlin]

I am writing this thread to offer an objective comparative analysis of some of the most popular ROMs available at this site. That being said, there are a lot of ROMs available on this site and I know that I have not included all of them, though I will be adding more in the coming days and weeks. If you have a ROM that you would like tested, please send me a link to your ROM via a PM and it will be included in the testing. I have tested ROMs in four different categories: WM 6.5 + Sense 2.1, WM 6.5 + Sense 2.5, WM 6.5.5 + Sense 2.1, and WM 6.5.5 + Sense 2.5. I have also included information for the stock Sprint ROM (WM 6.1 + Manila 2.1). I hope to add some Titanium only builds to the mix in the near future.

Testing Methods

All ROMs were tested in the same manner which includes the following: Each ROM was flashed from Bootloader and allowed to run through customizations while still connected to the computer. Upon reboot, the phone was disconnected from the computer and available RAM and ROM were recorded as well as OS and Manila versions. Note that the available RAM was documented while Manila/Sense was active. Next the device was connected back to the computer and SPB Benchmark was installed. There were a few ROMs in which I was unable to install SPB Benchmark. For most of these ROMs, I received ActiveSync errors which is denoted by “No AS Connection,” and one ROM failed while running the benchmark test. The benchmark tests included everything but the “Built-in Applications” because the tests kept freezing while running the MS Word and PIE analysis. Thus, the test do not include everything that was tested on the IPAQ which is SPB’s standard device. Nonetheless, all the ROMs here were tested in the same manner so the benchmark scores are relevant. Finally, after the benchmark test were complete, the device was shut off so that I could document the boot time. On each trial, with the phone turned off and unplugged from a power source, I turned the phone on while at the same time starting a stopwatch (G-Watch). Time stopped when Manila/Sense finished loading.

*Note: the pagepool settings for each of the ROMs are the default that is set by the chef, i.e. I did not set the pagepool settings to be equal.

NEW RESULTS ROM_Stats_5_15.pdf

I am sure you will notice that there are fewer categories this time around. I have noticed that most chefs have switched to building primarily Sense 2.5 ROMs. Also, other chefs have left to other phones or platforms so there are fewer ROMs for that reason. This update is meant to include only ROMs that are currently in development. I realize that I have missed a few (including the ROM I cam currently using) and will be doing a few more to add to this list this weekend.

WM 6.5.5/M 2.5
Most RAM- MightyROM Full Force
Most ROM- Zone Chief's Choice
SPB Benchmark- MightyROM Full Force
Processor Benchmark- MightyROM Full Force
Shortest Boot Time- Icey ROM

WM 6.5/M 2.5
Most RAM- Lead ROM
Most ROM- JD's
SPB Benchmark- MightyROM Official
Processor Benchmark- MightyROM SE
Shortest Boot Time- Lead ROM

BEST OVERALL
Most RAM- Lead ROM
Most ROM- SuperROM
SPB Benchmark- SuperROM
Processor Benchmark- SuperROM
Shortest Boot Time- Blackout ROM


OLD RESULTS
Results (see attached .pdf for full table of results) ROM Comps Data.pdf

WM 6.5.5/M 2.1
Most RAM- Mighty ROM Love
Most ROM- Mighty ROM Love
SPB Benchmark- MalROM v1.5b6
Processor Benchmark- MalROM v1.5b6
Shortest Boot Time- Mighty ROM Love

WM 6.5.5/ M 2.5
Most RAM- Baseline 002 RC
Most ROM- Baseline 002 RC
SPB Benchmark- MalROM Beta v1.6
Processor Benchmark- Baseline 002 RC
Shortest Boot Time- Baseline 002 RC

WM 6.5/ M 2.1
Most RAM- CFAY 2.0 RTM
Most ROM- MalROM v1.5b6
SPB Benchmark- MalROM v1.5b6
Processor Benchmark- MalROM v1.5b6
Shortest Boot Time- Silence_12

WM 6.5/ M 2.5
Most RAM- Medium Hill 2.5
Most ROM- MR. X 2.5
SPB Benchmark- MR. X 2.5
Processor Benchmark- MR. X 2.5
Shortest Boot Time- Mighty ROM War

Titanium
Most RAM- Baseline001
Most ROM- JD's Clean 6.5
SPB Benchmark- MalROM 6.5
Processor Benchmark- MalROM 6.5
Shortest Boot Time- JD's Clean 6.5


On a final note, I am able to run these test right now because I am not using my TP2 as my primary device though I will be again in a couple of weeks. That being said, I will do what I can to keep this updated until I start using my TP2 on a daily basis. When I switch again to another phone I will resume testing with another round of updated ROMs. I hope you enjoy the results and find the information useful. If you have any suggestions of things to add, remove, or do differently, let me know.

Here is an example of what the benchmark tests consist of:
Benchmark Examples.pdf