Analysing and Approving of Top Scores

[Adrena1in]

Hi everyone. Seems the debate about score approval has kicked off again. Have a few thoughts about improving the system for the World's Top few, and wondered if others have any particular views or opinions they'd like to share to help make the score-approval process at the top level a bit more stringent and effective.

Let me first just explain why I approved Paul's recent 15.5k;

1) He did several scores in the 15.3k to 15.5k range, and though I gather they all looked to be about 15.4k on the analyser, surely it's just as likely the 15.5k was genuine as the 15.3k was a low counter-error and the analyser was reading them all a bit low?

2) He scored 19k on the Raptor, and it's highly unlikely you can do this if you're not scoring well over 15k on the Plastic.

That was enough proof for me, especially as his 15.5k sounded just about exactly the same to me as a 15.5k from Duane Cash.

But maybe we should have a sort of panel of judges whenever someone submits a World Top 10 score or something like that? We all know that counter-errors happen, and we all know that the Powerballalyser on one machine can give a vastly different reading to itself running on another machine.

A useful method would be for people to submit several scores to back up a record-breaking submission. If three or four people are analysing three or four separate scores from someone, it's far more likely we'll be able to determine which scores are probably genuine and which scores are not. And all the scores submitted needn't be near the record score...in fact it might be better if they're not. For example, someone submitting a 15.5k score could then also submit a 15k and a 14.5k, done at the same time and in the same conditions, to act as controls to help in the analysis.

Thoughts? I know this isn't really going to work for the occasional top scorers who come out of the woodwork with a single spin and don't come to these forums, but at least it might help with the regulars here who are making their way towards the top.

[mroek]

We all know that counter-errors happen, and we all know that the Powerballalyser on one machine can give a vastly different reading to itself running on another machine.

For realtime analysis, it is possible that one may get different results on different machines, but vastly different? If results are vastly different, then there is something wrong with the machine(s) (could be hardware/drivers etc).

If you take one prerecorded file and analyze that file on different machines, you will get the exact same results.

Regards
-√òyvind

[Adrena1in]

If you take one prerecorded file and analyze that file on different machines, you will get the exact same results.

Thanks, I had always assumed that would be the case, but a while back Shazam achieved several 14.5k to 15k scores that his PC said were fine, but when I analysed the videos they showed as around 500rpm less. What was really odd is that he then analysed my 14.9k and said it showed up at nearer 15.4k on his machine. This was from the same video file, (though, granted, the way we both created the sound file from the video file is perhaps different...I *certainly* would expect Powerballalyser to give the same results on two PCs for the same sound file.)

Question is, what's the best way to create a sound file? It's *that* which is probably causing the spurious readings sometimes. Live Scan results are perhaps the better option here.

[pacman]

i agree lets have a panel of lets say six people who will have to look at all the factors of top ten spin ie video,sound history ect.
all though it could be on all scores over 15k.
and then they can approve scores or not.
you can put my name down but i wouldnt be able to look at own records.
we could vote for people.
this web site seems to be run by us and not admin at moment well done to ltlftc for doing video link page .
any thoughts from the owners and people who run this site ?????
oh and does the raptor counter fit a normal ball and is it a better improved counter.
pacman

[_AM]

I decided to register and make a few comments when I saw this:


and we all know that the Powerballalyser on one machine can give a vastly different reading to itself running on another machine.

Is that a fact?

I personally have been using PB-zer for over a year now, and ran it on some 10-20 different PCs -- desktops and notebooks, new, old, some very old, and I have *never* found its output to be different (so I would notice) to what I get on my own PC. And, yes, I always use realtime capture. Nor did I ever come across a discussion of this happening on Russian or this PB forums.

If this is such a well-known fact indeed, how come it was not discussed even once in this forum in the past ~15 months or so (not to my knowledge, at least)? How many people have ever encountered PB-zer giving vastly different reading on different PCs like you?

On the other hand, I can easily imagine a scenario which would yield exactly what you're talking about. And I'll tell you more, I could even "prove" that PB-zer is such complete and utter unreliable crap that it can give vastly different reading (as vastly as you want) on *any two PCs* you give me. :)


For realtime analysis, it is possible that one may get different results on different machines,

:) This is admirable, Oyvind, but to be honest, I fail to see how even wrong realtime analysis (by which I mean, the source of error is PB-zer itself) could be a practical possibility given my experience with it.

During realtime analysis, PB-zer doesn't talk to the sound chip directly, right? In case of capture from a file being played back, what it does is process data which comes from codec and is then sent to chip's DAC. In case of microphone capture, it uses data provided by soundchip's driver to OS kernel. What is important to note here, in both cases PB-er acts just like any other application on your computer.

Now, it's possible indeed that particular version of codec installed on a PC can be buggy in such a way that any (or very particular) file you play on the PC it decodes so that the rate is slightly different from reality, all sounds now having slightly different pitch. Ditto soundchip driver when it comes to, say, mic recording. Ditto -- theoretically, never heard of faulty quartz crystals -- sound hw, when ADCs and DACs are clocked not at the nominal rate they are supposed to be clocked at (e.g. 44.1 kHz) but something a tad lower or higher. But whenever anything of this is true, there would be tens, possibly hundreds of people reporting this in forums etc., for the problem would take place in *any* software used, with *anyone* using a copy of this codec/driver or soundcard, and thus it's highly unlikely to remain little-known and unfixed.

Of course, you can still argue that it's PB-zer which must be somehow buggy -- and I can't prove that it's not -- but hey, I'd bet anything you dare me on that on any two PCs where PB-zer gives vastly different reading, the pitch of the sound being captured would be just as vastly different as well. :)

[mroek]

:) This is admirable, Oyvind, but to be honest, I fail to see how even wrong realtime analysis (by which I mean, the source of error is PB-zer itself) could be a practical possibility given my experience with it.

During realtime analysis, PB-zer doesn't talk to the sound chip directly, right? In case of capture from a file being played back, what it does is process data which comes from codec and is then sent to chip's DAC. In case of microphone capture, it uses data provided by soundchip's driver to OS kernel. What is important to note here, in both cases PB-er acts just like any other application on your computer.
I can confirm that Powerballalyzer is a pure WIN32 api application (no wrappers, like MFC or anything), and to capture sound it uses the standard Windows Mixer api, like most other applications that need to deal with sound input/capture. If the realtime data is inaccurate, then the inaccuracy has nothing to do with Powerballalyzer itself.

I agree that errors are quite unlikely, but I can't say for certain that it cannot happen, so I am just being cautious here.

Regards
-√òyvind

[LTLFTC]

I think from an approval perspective;
You should show the counter to the camera while the ball is spinning, press clr to show the existing high score and then a new lower score. This will negate the occasions when the ball shows the previous high score straight after you have started it.
If you don't look at the ball after you have pressed clr you could miss this and think you have done better than you have.

As for pBLyzer. I don't understand all the technical jargon, but I do know for my camera on my computer recording live runs it is amazingly accurate. I can guess my score within 50 revs usually. It stays the same after I upload it to youtube and analyze it. (Though I keep getting that stupid delay any you techno dudes help out there.) I have also run one of my hand piece motors which has a speed control and it is very accurate for that as well. And I never get any difference on different computers either


I personally have been using PB-zer for over a year now, and ran it on some 10-20 different PCs -- desktops and notebooks, new, old, some very old, and I have *never* found its output to be different (so I would notice) to what I get on my own PC. And, yes, I always use realtime capture. Nor did I ever come across a discussion of this happening on Russian or this PB forums.

On the other hand, I can easily imagine a scenario which would yield exactly what you're talking about. And I'll tell you more, I could even "prove" that PB-zer is such complete and utter unreliable crap that it can give vastly different reading (as vastly as you want) on *any two PCs* you give me. :)
I can't decide if you like it or not. Please do present the scenario for us to show it is crap it will help in score verification.

But maybe we should have a sort of panel of judges whenever someone submits a World Top 10 score or something like that? We all know that counter-errors happen, and we all know that the Powerballalyser on one machine can give a vastly different reading to itself running on another machine.

A useful method would be for people to submit several scores to back up a record-breaking submission. If three or four people are analysing three or four separate scores from someone, it's far more likely we'll be able to determine which scores are probably genuine and which scores are not. And all the scores submitted needn't be near the record score...in fact it might be better if they're not. For example, someone submitting a 15.5k score could then also submit a 15k and a 14.5k, done at the same time and in the same conditions, to act as controls to help in the analysis.

Thoughts? I know this isn't really going to work for the occasional top scorers who come out of the woodwork with a single spin and don't come to these forums, but at least it might help with the regulars here who are making their way towards the top.Sound like a good idea.

[Adrena1in]

I personally have been using PB-zer for over a year now, and ran it on some 10-20 different PCs -- desktops and notebooks, new, old, some very old, and I have *never* found its output to be different (so I would notice) to what I get on my own PC. And, yes, I always use realtime capture. Nor did I ever come across a discussion of this happening on Russian or this PB forums.

I will amend my original statement and say that Powerballalyser *can* give different readings on two PCs, but only when third party software has been used to create sound files from a video. I go back to the situation where Matthieu Tauzin was consistently registering 500 or 600 rpm more when he used Powerballalyser. I don't think it's because of the Analyser itself, but because of other factors used to get .WAV files or what-have-you. (I never do real-time captures myself. I always convert my video to a MONO .WAV file and analyse that. That's always given accurate readings as far as I can tell. How Matthieu did his readings is unknown to me.)

And the Admins haven't forsaken us completely by the way. What with various things, the RPM guys have been having a very busy few months of it lately. But Enda's been in touch and is back to concentrating on the website and the scoreboards, so fingers crossed there'll be something new on the horizon soon.


Oh, and welcome back Andy. I hope you can manage to stay longer this time. :)

[Adrena1in]

i agree lets have a panel of lets say six people...

Plus I wasn't particularly intending there to be a fixed panel, and I don't know if it needs to be as many as six people. I was thinking that when a new score appears, as long as the person's a regular here they could perhaps post three Plastic runs and a Raptor run (if they have a Raptor). If perhaps three people could analyse each score, using whatever methods they normally use, and then post a positive or negative response about the record score, then it would just help capture the occasional unobvious counter-error.

[hugh495]

i think we should use this: http://www.youtube.com/watch?v=5M8Ra58xCso
seriously though that is brilliant
and not to go to off topic but oh my god
http://www.youtube.com/watch?v=Dg26hfSoRUk
so the normal counter still works at 18.5K

[Adrena1in]

It's not that portable though, Hugh!! ;)

[mroek]

and not to go to off topic but oh my god
http://www.youtube.com/watch?v=Dg26hfSoRUk
so the normal counter still works at 18.5K

Actually, it doesn't. The speed in that video is WAY over 18.5k, more like 25k judging from the sound in the video. The counter on that specific Powerball certainly has a problem with this speed.

Regards
-√òyvind

[_AM]

and not to go to off topic but oh my god
http://www.youtube.com/watch?v=Dg26hfSoRUk
so the normal counter still works at 18.5K

didn't check the video yet, but if it's compressed air as it says in the description, then a bunch of crazy Russians did it about a month earlier: http://www.youtube.com/watch?v=TSfmKpYOZ_0
I personally enjoy the laughter at the very end. There's something almost satanic about it. :)


I can't decide if you like it or not. Please do present the scenario for us to show it is crap it will help in score verification.

Oh, I was being extremely ironic Rob; wrt trustability PB-zer is just fine as long as I'm concerned. As for the scenario, I don't see how awareness of it could help fight fake achievements in any manner. The only party to definitely benefit from the description is those who want to boast amazing speeds rather than work hard; for them it would be a golden opportunity to look better than they really are.



I will amend my original statement and say that Powerballalyser *can* give different readings on two PCs, but only when third party software has been used to create sound files from a video.

Then I don't exactly get the point, sir. Are you saying now that PBzer was found to give different readings for two different files created from the same video (possibly using different audio extractors)? If that's the case, then why not just create two files using these two extractors and run them through PBzer? That would clarify things a lot (since your original point was about *PBzer* giving different readings to itself on different PCs -- not about audio extractors found to alter pitch in extracted audio, or about computers found to playback the very same file at different pitch).

Besides, I would really like to get an idea how many people experienced something similar. And on how many occasions *you* found PBzer to work differently on different PCs. Presuming you've been using PBzer for over 3 years to validate speed runs, you could have analyzed hundreds of videos in the meantime. So how many occasions were there? Was it tens? A few? Or just Matthieu Tauzin?

Don't get me wrong: if this is something "you all know", I'd surely like to know it as well, and, what's more important, hopefully find the reason and deal away with this problem once and for all.


Oh, and welcome back Andy. I hope you can manage to stay longer this time.

LOL, touche!!! Thanks for the welcome Tim. :)

[LTLFTC]

and not to go to off topic but oh my god
http://www.youtube.com/watch?v=Dg26hfSoRUk
so the normal counter still works at 18.5K
I would say no, I reckon that was around a 19640 run.

[Adrena1in]

You guys aren't taking Hugh literally. He said the counter still works at 18.5k, which apparently it does...just not higher! ;)

[mroek]

I posted a reply that somehow got lost, but judging from the sound in that video, the speed is WAY over 18.5k, it is more like 25k. If anything, it proves that the counter on that specific powerball does not handle such speeds.

[Adrena1in]

Yes, I'm sure I remember the RPM Sports guys saying the counter had to be improved to cope with the speeds the Raptor produces. But the standard counter does still work to around 18.5k, as I said...just not much higher. (Perhaps those guys will do another such test with a Ratpr counter.)

(In that video, I'm glad one of the axles didn't suddenly bite on the shell as at that speed it could've flung the rotor across the room!!)