Additional statistics and graphical analysis of SuperPi/PiFast scores

[Gautam]

What's up guys. :)

I was chatting some SuperPi with a friend last night and figured this might be a neat thing for you guys to implement if you have any spare time. (what's that?)

As you can see from the graphs below that Vapor whipped up, time and CPU speed are inversely proportional. With a trendline like this, or even without, its easy to visually predict what sort of time you'd end up with at a given CPU speed. This would work equally well for SuperPi 32M and PiFast.

Whenever people ask how they're times and scores stack up, I always point em to hwbot. It'll be even reasier to see how one compares to others or what one should be expecting at certain clocks graphically like this.

Additionally perhaps we could consider displaying performance products for all scores and showing how someone's scores compare in efficiency to others.

I think this stuff should be fairly easy, for these graphs Vapor and I each spent a couple of hours collecting data points off the hwbot 1M ranking. Would be nice to see the same process automated. :p

Thanks again for all your continued hard work.

[richba5tard]

I love charts, and technically they're pretty easy to implement. The hard part is picking out 'good' samples. As hwbot receives 300 to 500 submissions a day, the result moderators barely have time whether essential info is correct (correct score, correct cpu/gpu and has validation), so there are quite a lot of scores which have wrongly matched cpu/gpu speeds.

There are multiple ways to solve this: more mods, better tools to detect these faulty scores for mods, or better algorithms for charts which detect bad results. We're working on the latter 2. For example, we just made the 'overclockability' chart better for videocards:
http://www.hwbot.org/quickSearch.do?hardwareId=GPU_337&name=Radeon+9800+Pro+%28498%29
(scroll down a bit). It will take some time before we are confident we can add more charts. Better 1 good chart than 10 poor ones. :)

[jmke]

those tweaked VGA charts are sweet!

http://www.hwbot.org/cewolf/;jsessionid=A2440B8E5F05FEBA355B204E32E32C11?img=1 843963543&width=600&height=300&iehack=.png

[richba5tard]

You can clearly see people like rounded numbers. :) 1000 and 1100mhz are the most popular memory speed.

666mhz gpu core speed is clearly very popular too. :D

[jmke]

too bad people are reporting WRONG clockspeeds for the 8800 series, as the clock increase by 27Mhz (approx) some speeds are not "real". http://www.madshrimps.be/?action=gethowto&number=4&howtopage=191&howtoID=72

[Gautam]

I love charts, and technically they're pretty easy to implement. The hard part is picking out 'good' samples. As hwbot receives 300 to 500 submissions a day, the result moderators barely have time whether essential info is correct (correct score, correct cpu/gpu and has validation), so there are quite a lot of scores which have wrongly matched cpu/gpu speeds.

There are multiple ways to solve this: more mods, better tools to detect these faulty scores for mods, or better algorithms for charts which detect bad results. We're working on the latter 2. For example, we just made the 'overclockability' chart better for videocards:
http://www.hwbot.org/quickSearch.do?hardwareId=GPU_337&name=Radeon+9800+Pro+%28498%29
(scroll down a bit). It will take some time before we are confident we can add more charts. Better 1 good chart than 10 poor ones. :)

What I was thinking was just having the charts created dynamically based off off of all submitted results. In fact, this would make reporting false scores even easier as outliers on a chart would get caught much quicker and more easily than if they were on a list. However the main benefit as I see it would just be in aiding in extrapolation and visually being able to see scaling.

Note for example the ever-so-slight curvature in the speed vs. time plot. Subtle, but its there, just like any good hyperbola. :p:

[jmke]

problem would be to make these outlined scores to be link-able and detectable in the chart :)

seeing the dot won't be a problem, find out from just looking at the chart WHAT result it is... a bit harder;)

[Gautam]

Yeah TBH in my ideal image, each dot links to the compare link for that score. That way if you'd see an outlier you could report it immediately.

But that might be pretty difficult to implement. :p

[richba5tard]

difficult and a HUGE page. The underlying html file would contain up to 10k compareurls. :)

Maybe a feat for paying members. :evil:

[Gautam]

Heh.

Even without any links you wouldn't need to have "cleaned" graphs. I still think that automatically generating the graphs would be fine. Outliers would still be easier to catch. I can't see any downside to having current results displayed graphically.

[richba5tard]

Here's a fine example:
http://www.hwbot.org/hardware.processor.statistics.do

Very interesting chart, but as good as all the time screwed up by bad results.

[jmke]

sexy

http://www.hwbot.org/cewolf/;jsessionid=9D726C4170B1F28D3E131E3594482391?img=-1967610801&width=800&height=700&iehack=.png

[Maxi]

The other way it could be done is simply showing pp in the statistics of each score, that would be a real nice addition and a quick reference to each board/setup/skill of each users result :)

[richba5tard]

pp? people nickname? :)

[Maxi]

Performance Product - used for Super Pi charts :)