PPCGeeks - VS Benchmark

PPCGeeks (http://forum.ppcgeeks.com/index.php)

- HTC Touch Pro (http://forum.ppcgeeks.com/forumdisplay.php?f=63)

- - VS Benchmark (http://forum.ppcgeeks.com/showthread.php?t=53217)

Comparison Benchmark

As requested, here you go. Spent hours flash and reflash my TP numerous times for this comparison. Hope it helps:

Fresh Benchmark
Flash > AutoReset+Init > Install VSBenchmark to Main Memory > Hard Reset > Run Benchmark (on level surface,all cables disconnected)

JUICY 47 Herman
2223 - 2111 - 2234 [GRAPHICS]
2573 - 2609 - 2613 [OTHERS]
4769 - 4769 - 4835 [JPEG]
1786 - 1786 - 1822 [GAMES]
1064 - 1054 - 1053 [SOUNDS]
2484 - 2464 - 2511 [SCORE]

MIGHTY v4.9.2
2321 [Graphics]
2227 [Others]
4613 [Jpeg]
1996 [Games]
0939 [Sounds]
2419 [SCORE]

KEVAN-K Loaded v1.2
2297 - 2268 [Graphics]
2640 - 2702 [Others]
4691 - 4732 [Jpeg]
1967 - 1969 [Games]
1142 - 0948 [Sounds]
2547 - 2523 [SCORE]
*Lots of useful apps

KEVAN-K Loaded v1.3
2294 - 2312 [Graphics]
2737 - 2625 [Others]
4779 - 4743 [Jpeg]
1989 - 1988 [Games]
0932 - 0943 [Sounds]
2546 - 2522 [SCORE]
*Lots of useful apps

CALKULIN v1.31 Clean
2248 - 2235 - 2247 [Graphics]
2496 - 2471 - 2503 [Others]
26174 - 26174 - 26174 [Jpeg]
1821 - 1813 - 1813 [Games]
1054 - 1057 - 1057 [Sounds]
6758 - 6750 - 6758 [SCORE]
* Very Responsive StartMenu
* JPG Score flies because it's broken -bug?

GC v2.1.4
2314 [Graphics]
2773 [Others]
4712 [Jpeg]
1982 [Games]
0939 [Sounds]
2544 [SCORE]
* TouchFlo not installed

SSK v1.4
2234 [Graphics]
2116 [Others]
26174 [Jpeg]
1800 [Games]
1057 [Sounds]
6676 [SCORE]
* Probably has the same display problem
* Still have verizon reference in the 'Sprint' ver :D

eRice - CLEAN
2286 [Graphics]
2724 [Others]
4865 [Jpeg]
2024 [Games]
0937 [Sounds]
2567 [SCORE]

I guess my fave overall for ready to use ROM is KevanK's. I really hope there is a Kitchen for it.
-V

Re: VS Benchmark

Merdins Rom V3.4
touchPro
config:Heavy, activesync srewed my pim with 2215contacts before test
Grafics:2230
others:2020
Jpeg:26174
Games:1801
Sound:1254
Final Score:6695

EDIT: my other results with kk1.3 rom were less than VreebieZ, but ran slightly better in some ways than this nicer looking one, my two favorites, so yes the test results do not reflect the experience of playing with the rom.
Boils down to preference...

Re: VS Benchmark

Guys, all this shows is that the benchmark isn't valid :)

Post #31 at least tried multiple runs on the same rom/hardware, and various roms on the same hardware.

The only way to make the benchmark meaningful is to control for every variable (ie, every reg setting that changes the score)

Once we have a set of reg settings that we all agree on, and a test procedure we all agree on, then we can generate meaningful numbers (looks like +/- 1% across trials for normal variance) and compare them. That will tell you which rom OS is performing better or worse - likely the results will vary just above the noise (maybe 5% spread.)

Me personally, I don't care ;) But seriously if you want to do this that's the way, and VreebieZ has generated the most useful data so far. Note how all his "good" results were in the ballpark of 2500 (+/- 50, or 2%) - you'd be REALLY hard pressed to notice one device being 2% faster then another as a user.

Re: VS Benchmark

I tested VSBenchmark with various reg settings and my score stayed between 2389 and 2396 on over 10 different tests. I agree after playing with this for hours it is worthless.

Re: VS Benchmark

I think this is great effort for sure. Anyone have linkage to that ATI Driver? Thanks in advance....
:thumbleft:

Re: Comparison Benchmark

Quote:

Originally Posted by VreebieZ (Post 686068)

How come I score in the 6K along with Caulkin but the rest score in the 2K area? Is that a typo? Sorry dont have time to read the thread. will catch up when I am not at work.

Also, have you tried this bench using the SPB app? Just curious.

EDIT: And where did you see the VZW reference?

Thanks!

EDIT2: Nevermind. i ddi the math. Ignore first question. Hehehe

Re: VS Benchmark

Quote:

Originally Posted by schettj (Post 686169)

Guys, all this shows is that the benchmark isn't valid :)

Thats not what it means at all. There is always a margin of error with experimentation which is why experiments are always done in multiple trials.

I think a better procedure would be to take three trials of each one and average the results. Lots of work though so I don't blame anyone for not doing it :lol:

Re: Comparison Benchmark

Quote:

Originally Posted by scrosler (Post 689722)

How come I score in the 6K along with Caulkin but the rest score in the 2K area? Is that a typo? Sorry dont have time to read the thread. will catch up when I am not at work.

26174 [Jpeg]

There is some "bug" in some of the custom roms that give a wildly high score on the Jpeg test, and end up giving you an impressively invalid total score

Re: VS Benchmark

2727 on MightyROM 4.9.4

Re: VS Benchmark

Quote:

Originally Posted by EtherealRemnant (Post 696503)

2727 on MightyROM 4.9.4

what subcategory? (graphics, other, etc)