Stunts Forum

Stunts - the Game => Stunts Chat => Topic started by: Duplode on April 01, 2012, 05:34:13 PM

Poll
Question: Which would be a better metric for the strength of a race?
Option 1: Combined strength of all pipsqueaks votes: 1
Option 2: Strength of top pipsqueaks votes: 2
Option 3: Strength of midfield votes: 1
Option 4: Some other function of pipsqueak strengths votes: 1
Option 5: Something entirely different (say, pipsqueak activity) votes: 2
Title: Race strength estimation
Post by: Duplode on April 01, 2012, 05:34:13 PM
Question: suppose we had a consistent SWR-style metric for estimating pipsqueak strengths at a given instant. In such a case, which would be the best way of deciding if one race was stronger than the other? If you vote for 2 or 3, please also tell what would count as "top" or "midfield".

(The motivation for that is that I'm doing some speculative work on rankings, and want to validate some assumptions. Expect news about that in the next few days...)
Title: Re: Race strength estimation
Post by: BonzaiJoe on April 01, 2012, 06:01:00 PM
I think the most obvious would be to ask "how much work (*skill*luck) is required to attain nth position in this race", for all n, but the weight diminishing along some curve, perhaps linearly, perhaps with a bias for podium spots (because they are discriminately valued by pipsqueaks).
Title: Re: Race strength estimation
Post by: Duplode on April 06, 2012, 02:47:22 AM
Let us continue the discussion... I feel that perhaps a good case can be made for the counter-intuitive strength-of-midfield option: while even in weaker races there are two or three top pipsqueaks fighting for the victory,  if you looked at a scoreboard and saw Ayrton in 6th place you'd instantly know that was an exceptionally hard race. (By the way, by midfield I mean, roughly, the region around 2/5 of the way down the scoreboard.)
Title: Re: Race strength estimation
Post by: zaqrack on April 06, 2012, 05:43:09 AM
yes, I would agree with that. Another factor to consider could be the gap between the average finishing time of the three podiumers against the average or median finishing time of the top 12

Or to unleash the power of the accumulated history date (unified data is available from 2004, so already more than 100 tracks!
- gap between the podiumers compared to historic averages of the same value
- the position of some "key" pipsqueaks on the scoreboard against their historic average position (which is shown on the profile page)

Title: Re: Race strength estimation
Post by: zaqrack on April 06, 2012, 05:48:55 AM
also worth to mention, that any of the above factors could only work after a race is finished or near the final days.
For a similar estimate in the first two weeks of a race we could rely on the number of changes on the leading position and the number of participants sending replays.
Title: Re: Race strength estimation
Post by: Chulk on April 06, 2012, 11:16:28 AM
Quote from: Duplode on April 06, 2012, 02:47:22 AM
if you looked at a scoreboard and saw Ayrton in 6th place you'd instantly know that was an exceptionally hard race.
This is not entirely true... I think I can achieve a 5th place regularly if I had enough time to race at Zakstunts. That doesn't necessarily mean that every race where I had not enough time/motivation was specially hard. On the other hand, my average position might be 10th because I lack the time to race. Then maybe I just have a quiet month at work/university, raced a lot and finish 5th. Would you say that race was specially easy?.

I think there's no accurate way to measure something that has so many variables like time, motivation, amount of drivers, driver's car skill, teammates (of course it's easier to have a top driver as a teammate than a newbie), even the moment of the year has an impact on race results.

If I had to do it, I would see how much can time be improved in a given moment. For example, I don't think Renato Biker's legendary Countach replay cannot be improved as much as any of Roy's winning replays.

Other possibility is comparing winning time against imaginary sum of fastest sections, but that could also be really innacurate in races like the one Alan Rotoi wo with Ferrari, 1st race of 2005 I think...
Title: Re: Race strength estimation
Post by: Friker on April 08, 2012, 02:03:19 PM
after thinking about it i came to conclusion that quality of a race has no correlation with quality of pipsqueaks. if you have to use the quality of pipsqueaks it's probably a number of top pipsqueaks actively participating. but i think it's better to measure it as "how hard was to take nth place". (probably first seven)
Title: Re: Race strength estimation
Post by: Duplode on April 08, 2012, 03:57:01 PM
Starting with Friker's: the problem is how to convert "how hard it was" into a number. I would have to disagree with you in that strength of involved pipsqueaks does have correlation with that, although of course it is far from a perfect one. Ideally, the measure would include how strong the replays were (see Chulk's reply); but for that one would need to either 1. have a perfect (!?) reference lap for every track (!!); or 2. have section data for all races and then rely on imaginary sum of fastest sections (and then be subject to distortions like Z47, as Chulk points out; not to mention multi-car ZakStunts makes this method significantly more unreliable). Done consistently, both alternatives would mean massive amounts of work for data collection; and, for that reason, I am more inclined to, at least as a first approximation, find ways of measuring race strength which do not require watching or driving replays.

The Chulk-with-a-listfiller v. Chulk-in-full-power problem is a hard one, too. If a pipsqueak drives most races below his/hers full potential, in most races it would be appropriate to use a strength-with-listfillers value. But how to deal with the races driven at full power? And how to distinguish these from a fluke result? (There might be a way that alleviates this problem; maybe factoring in both a ranking counting all races by a pipsqueak and another including only his/hers best x% results.)

Zak's suggestions could be useful, and fun to play with. The "how hard it is" approach of Friker and BJ leads to some interesting, and possibly feasible, considerations on computing strengths for a specific position in a race instead of for the race as a whole.

Later I will post a sketch of what the calculation scheme I am planning looks like ATM. And finally: it is all just numbers, I know. But numbers can be amusing, too  :)



Title: Re: Race strength estimation
Post by: Friker on April 08, 2012, 06:12:33 PM
hmm.. ok some of my thoughts: (i did not write them earlier because some seemed to be off-topic)
-ordering should be symmetric and transitive ("a<b implies b[not less then]a" and "a<b and b<c implies a<c") (the third rule is irreflexivity but it's not as important in this situation)
-there is no unbiased way how to tell pipsqueak A is better then pipsqueak B: a day before yesterday i read some article where was used term "dominate" and it means "A dominates B when A is quicker then B in all cases" (quicker because they were comparing the speed of algorithms). ok, this is nice idea but algorithms always run as fast as they can. pipsqueaks do not. even if we want to substract worst places for each pipsqueak, this is not correct because of some pipsqueaks as zak. but still the substraction is a good heuristic. probably a few first races should not count to that domination's calculation. (some kind of newbie status)
-another way how to calculate something is: take the 5th (or 6th) place and calculate percentual value of other places. (for example 1st time was 60sec 5th time was 90sec - a value of 1st time is -33.33% which could be converted to -33.33pts) 5th place because its somewhere middle between 1 and 10 which are only relevant times. places below 10 would not be counted.
ok that was about A is better than B. what about race R is stronger then race S?
-hm? and what does it means - race R is stronger then race S? this is far more complex problem then it seems to be. i give you my off-topic opinion about races:
[more off-topic]
i play (frisbee) ultimate. we train, we play, we have tourneys. also we compete a lot. but a good turney means there is fun at a field and also at a party. And also a good turney has to have a good breakfast. :) the fun is more important than the final place. so a strength of a race is one thing but the fun around is something else.
[/more off-topic]
-i still think that BJ's "how hard is to take nth place" is still the most accurate formulation of what the strong race is.
-another way could be  through "what is the sum of being dominated of pipsqueaks in top N" where N should be 6-10 (really good races will have this number 0, still good races will have somewhere around 4-10, with bigger N there is more accent to worse pipsqueaks)
-the straightforward way is to count how many good pipsqueaks were participating in the race - here is a place to take that percentual thing comes into play

there are many other approaches described and probably a way better than this one but we don't know about them.. (so what we are doing here is probably reinviting a wheel)
Title: Re: Race strength estimation
Post by: Duplode on April 09, 2012, 03:37:17 PM
P.S.: Your comments about Ultimate made me realized I still had not mentioned one important point about what I have in mind. Ultimately, measuring race strength is not an end in itself. Rather, the goal is using race strengths to weigh the points earned in the races when calculating, for instance, average score rankings for pipsqueaks. For those of you who remember Mark's SWR, the general idea somewhat resembles its "quality points", but calculated in a way that makes better use of the available historical data.
Title: Re: Race strength estimation
Post by: CTG on April 09, 2012, 04:04:22 PM
I think it's useless to define the strength of a race - it depends on too many factors. Now telling a single one: personal motivation levels. When BJ finishes 9th, you just can't take it seriously. And how can you decide that Ayrton would be able to drive 2, 5 or 10 seconds better than the winning time?

As for me, you mustn't calculate the strength of the race - you should rather feel it.
Title: Re: Race strength estimation
Post by: Duplode on April 09, 2012, 04:56:20 PM
Valid points, but you don't need to read too much into it. All I want with these ideas about "race strength" is to have a factor which accounts for  the intuition that , e.g. your victory in Z49 should have more weight than the one in Z73. It doesn't have to be definitive or perfectly accurate. In fact, it is not even necessary that such race strength numbers compose a meaningful ranking on their own - as long as the factors are reasonable, the relative order of the races is of little consequence.

(To make it a bit clearer: suppose we adopted the current F1 system - 25, 18, 12... -  for base scores. If the "strength factor" was 1.1 for Z49 and 0.8 for Z73, your victories in them would translate into 25*1.1 = 27.5 and 25*0.8 = 20 points, respectively.)
Title: Re: Race strength estimation
Post by: CTG on April 09, 2012, 07:25:04 PM
Quote from: Duplode on April 09, 2012, 04:56:20 PM
Valid points, but you don't need to read too much into it. All I want with these ideas about "race strength" is to have a factor which accounts for  the intuition that , e.g. your victory in Z49 should have more weight than the one in Z73. It doesn't have to be definitive or perfectly accurate. In fact, it is not even necessary that such race strength numbers compose a meaningful ranking on their own - as long as the factors are reasonable, the relative order of the races is of little consequence.

(To make it a bit clearer: suppose we adopted the current F1 system - 25, 18, 12... -  for base scores. If the "strength factor" was 1.1 for Z49 and 0.8 for Z73, your victories in them would translate into 25*1.1 = 27.5 and 25*0.8 = 20 points, respectively.)

No problem with the math part, it will be a great work if you create this system. But I think it's a little bit useless. Maybe the problem is with me: I can't think on Stunts racing as a major part of my life - not like in the past 10 years (LOL, I'll be "10 years old" on Friday). Nowadays I drive only when I have time and mood.
Title: Re: Race strength estimation
Post by: zaqrack on April 10, 2012, 01:02:31 AM
Quote from: CTG on April 09, 2012, 07:25:04 PM
Nowadays I drive only when I have time and mood.

So you have finally also achieved the state of Stunts-Zen :D
Title: Re: Race strength estimation
Post by: Duplode on April 10, 2012, 03:01:38 AM
Quote from: CTG on April 09, 2012, 07:25:04 PM
No problem with the math part, it will be a great work if you create this system. But I think it's a little bit useless. Maybe the problem is with me: I can't think on Stunts racing as a major part of my life

No worries - in a way, you're right. What makes a ranking useful, anyway? Only if the positions are worth anything (as in SWR-ISM), or else if you are going to brag about your scores (which I surely won't :D). To me, these plans are both a programming exercise about [suppressed long-winded explanations about the tricks I am trying to learn/apply]  and a way of playing with a fun question that roams around the community since time immemorial.

Quote from: CTG on April 09, 2012, 07:25:04 PM
LOL, I'll be "10 years old" on Friday

Hey, I wasn't aware your "birthday" is in April too  :)
Title: Re: Race strength estimation
Post by: CTG on April 10, 2012, 07:27:41 AM
Quote from: Duplode on April 10, 2012, 03:01:38 AM
Quote from: CTG on April 09, 2012, 07:25:04 PM
No problem with the math part, it will be a great work if you create this system. But I think it's a little bit useless. Maybe the problem is with me: I can't think on Stunts racing as a major part of my life

No worries - in a way, you're right. What makes a ranking useful, anyway? Only if the positions are worth anything (as in SWR-ISM), or else if you are going to brag about your scores (which I surely won't :D). To me, these plans are both a programming exercise about [suppressed long-winded explanations about the tricks I am trying to learn/apply]  and a way of playing with a fun question that roams around the community since time immemorial.

Quote from: CTG on April 09, 2012, 07:25:04 PM
LOL, I'll be "10 years old" on Friday

Hey, I wasn't aware your "birthday" is in April too  :)

13th April 2002, as far as I remember. Damn, I was sooo young...
Title: Re: Race strength estimation
Post by: Chulk on April 13, 2012, 10:43:26 AM
Quote from: CTG on April 10, 2012, 07:27:41 AM
13th April 2002, as far as I remember. Damn, I was sooo young...
Is this the day you joined the forum or your first Zakstunts replay?

BTW: HAPPY B-DAY!
Title: Re: Race strength estimation
Post by: CTG on April 13, 2012, 12:58:04 PM
HAHAHAHA thank you! :D
Title: Re: Race strength estimation
Post by: Chulk on April 13, 2012, 06:34:57 PM
Quote from: CTG on April 13, 2012, 12:58:04 PM
HAHAHAHA thank you! :D
You didn't answer my question, mister...
Title: Re: Race strength estimation
Post by: CTG on April 13, 2012, 08:51:05 PM
Replay, ZCT 13.
Title: Re: Race strength estimation
Post by: Chulk on April 14, 2012, 03:12:35 AM
Then mine is somewhere around February 2005. Although that lap wasn't valid or something because I finished with 9:99.99... don't remember what happened. First VALID replay was in March, no idea about the day I sent it.
Title: Re: Race strength estimation
Post by: alanrotoi on April 14, 2012, 07:35:44 AM
When was created your first ghost?  :-* :P
Title: Re: Race strength estimation
Post by: CTG on April 15, 2012, 10:49:15 PM
Quote from: alanrotoi on April 14, 2012, 07:35:44 AM
When was created your first ghost?  :-* :P

Suck my Indy!
Title: Re: Race strength estimation
Post by: Overdrijf on April 20, 2012, 02:16:28 PM
I'd say the best metric is the strength of the drivers just below the podium, numbers 4-8 or so. Even if there are two or three guys fighting an awesome battle at the top you never know how the rest of the field turns out. If there is a strong subtop it doesn't really matter if the real top is battling extra hard, the race is a good one anyway.

[/ontopic, please do continue sucking his Indy]
Title: Re: Race strength estimation
Post by: CTG on April 20, 2012, 02:44:02 PM
What about time hiding? Imagine two possibilities.

Z437 (whatever)

1 Inavoeg 45.50
2 Roy Wiegerinck 45.55
3 Mark Nailwood 45.65

...but all of them sent their best during the quiet days, Ben Snel was leading for 20 days with 52.70.

Same track, paralel universe:

1 Inavoeg 46.00
2 Roy Wiegerinck 46.75
3 Mark Nailwood 47.20

Not as tight, not as fast as the former one, but the leading time board is quite long (7 different leaders, milliseconds war during the whole month) and only smaller improvements took place during the quiet days.

Which one is stronger and more entertaining?

Title: Re: Race strength estimation
Post by: Duplode on April 20, 2012, 03:26:15 PM
Quote from: Overdrijf on April 20, 2012, 02:16:28 PM
I'd say the best metric is the strength of the drivers just below the podium, numbers 4-8 or so. Even if there are two or three guys fighting an awesome battle at the top you never know how the rest of the field turns out. If there is a strong subtop it doesn't really matter if the real top is battling extra hard, the race is a good one anyway.

Nice way of putting that argument  ;)

Quote from: CTG on April 20, 2012, 02:44:02 PM
What about time hiding? Imagine two possibilities.

Z437 (whatever)

1 Inavoeg 45.50
2 Roy Wiegerinck 45.55
3 Mark Nailwood 45.65

...but all of them sent their best during the quiet days, Ben Snel was leading for 20 days with 52.70.

Same track, paralel universe:

1 Inavoeg 46.00
2 Roy Wiegerinck 46.75
3 Mark Nailwood 47.20

Not as tight, not as fast as the former one, but the leading time board is quite long (7 different leaders, milliseconds war during the whole month) and only smaller improvements took place during the quiet days.

Which one is stronger and more entertaining?

Well, if heavy time hiding demotivated midfielders and led to weak replays just below the podium I agree the race would be less "strong" in the limited sense I am talking about here  (to some extent, that applies to races with "impossible" sent very early in which no one bothers to challenge the leaders). This situation, too, suggests that calculations involving not only the final positions but also final times could improve things; as well as Friker/BJ's strength-per-position ideas. (Yesterday I rewrote the algorithms I am using in my tests in a simpler way; so now it is feasible to try adding some more sophisticated features to it.)

I will return to this topic tonight with some actual numbers, so that you can judge how (im)plausible this whole thing is  ;)
Title: Re: Race strength estimation
Post by: Duplode on April 22, 2012, 12:51:43 AM
Okay, time to show some concrete stuff. What you see below is a plot of race strengths for every ZakStunts race up to Z128. The calculations were done in pretty much the way I conceived them about a month ago - later I will try to incorporate some of your suggestions to see what happens. As CTG and others pointed out, there is not that much meaning in these results per se, but still you can tell me how reasonable they appear to be.

(http://i49.photobucket.com/albums/f283/Duplode/stg1.png) (http://i49.photobucket.com/albums/f283/Duplode/stg1.png)

(Clicking on the image will open a larger version.)
Here are the plotted strength values, first in chronological order and then sorted in descending order of strengths.


1       "C01"   0.9657091867078631
2       "C02"   0.8141785631288354
3       "C03"   0.7453159572652599
4       "C04"   0.7922908507387288
5       "C05"   0.6753799676843679
6       "C06"   0.7092608821171026
7       "C07"   0.8252295579725029
8       "C08"   0.8282797572116257
9       "C09"   0.9389405196765955
10      "C10"   1.0936190954242895
11      "C11"   1.2363536177595087
12      "C12"   1.1800993388722651
13      "C13"   1.1203658777932923
14      "C14"   1.2074147198905056
15      "C15"   1.182451898551215
16      "C16"   1.1588507034574764
17      "C17"   1.1676369870653622
18      "C18"   1.1052430073574848
19      "C19"   1.2080674589235236
20      "C20"   1.0504654861397411
21      "C21"   1.2431761847373568
22      "C22"   1.1340134845563254
23      "C23"   1.0614424296335876
24      "C24"   1.175037589853663
25      "C25"   0.972694161917751
26      "C26"   1.2203471578902352
27      "C27"   1.0312946098104023
28      "C28"   1.085988792445158
29      "C29"   1.1821913019831896
30      "C30"   1.02883008931889
31      "C31"   1.0943889053459661
32      "C32"   1.0845407480432159
33      "C33"   1.0555699446620956
34      "C34"   1.1960932075802668
35      "C35"   1.1837484780107108
36      "C36"   1.0958282811902105
37      "C37"   1.1034434409399259
38      "C38"   1.1173039921849235
39      "C39"   1.1475132728957917
40      "C40"   1.1061285013485165
41      "C41"   1.11006151888716
42      "C42"   1.0643876605531166
43      "C43"   1.094279903873983
44      "C44"   1.077618594281165
45      "C45"   1.103890752144628
46      "C46"   1.119112793710933
47      "C47"   1.14612193885565
48      "C48"   1.0726565224848388
49      "C49"   1.0775341436490882
50      "C50"   0.9160096091266053
51      "C51"   1.2269584736225978
52      "C52"   1.2051430846019162
53      "C53"   1.0107860881772848
54      "C54"   1.228220214010276
55      "C55"   1.1755257049670735
56      "C56"   1.1841688404382151
57      "C57"   1.1004882057221215
58      "C58"   1.050609109559689
59      "C59"   1.186922711687243
60      "C60"   1.243762215547041
61      "C61"   1.2190762100675552
62      "C62"   1.0447622124238727
63      "C63"   1.206750047819268
64      "C64"   0.9953957458452458
65      "C65"   1.1018134545003677
66      "C66"   1.0935899101521043
67      "C67"   0.957937805004983
68      "C68"   1.0331664619736538
69      "C69"   1.017739584620091
70      "C70"   0.8644722880290338
71      "C71"   0.6817503422504274
72      "C72"   0.917570198503481
73      "C73"   0.664522740721276
74      "C74"   0.7213651251811808
75      "C75"   0.6646188665720576
76      "C76"   0.6515001369508422
77      "C77"   0.6437625284391406
78      "C78"   0.6810065009509227
79      "C79"   0.909509231033356
80      "C80"   0.8540853663893501
81      "C81"   0.9297857135310638
82      "C82"   1.013651690012841
83      "C83"   0.9451541011596529
84      "C84"   0.8958702296865297
85      "C85"   0.9541623988424536
86      "C86"   1.0301343609824418
87      "C87"   0.9757621925074781
88      "C88"   0.9723272981150329
89      "C89"   0.9467397799776766
90      "C90"   0.9772709783527275
91      "C91"   0.9053582079598089
92      "C92"   1.029297759021717
93      "C93"   0.8432195616244242
94      "C94"   0.9987223306578578
95      "C95"   0.9580879953822644
96      "C96"   0.9953046898125949
97      "C97"   0.9970201633770771
98      "C98"   1.040630483348009
99      "C99"   0.9488985373983424
100     "C100"  1.3818428622781516
101     "C101"  1.1309067391403302
102     "C102"  0.9196914991385376
103     "C103"  0.9706867284281467
104     "C104"  0.9509179348993253
105     "C105"  1.0321136466012524
106     "C106"  0.8671373586094492
107     "C107"  0.9270645491610486
108     "C108"  0.9150850552327898
109     "C109"  0.8761377544007346
110     "C110"  0.9574757518387493
111     "C111"  0.9843195772194393
112     "C112"  0.8568413904544915
113     "C113"  1.0461070026034618
114     "C114"  1.016027543932544
115     "C115"  0.9255790594836256
116     "C116"  0.8405878125873606
117     "C117"  0.6802089951307128
118     "C118"  0.7953127915092747
119     "C119"  0.8351134232054357
120     "C120"  0.636479016610417
121     "C121"  0.7494933271150035
122     "C122"  0.9273912941259649
123     "C123"  0.8216658812447498
124     "C124"  1.0065077908416147
125     "C125"  0.9667008689224187
126     "C126"  0.9841042932870333
127     "C127"  1.0173974790741993
128     "C128"  0.9803232497592262



1       "C100"  1.3818428622781516
2       "C60"   1.243762215547041
3       "C21"   1.2431761847373568
4       "C11"   1.2363536177595087
5       "C54"   1.228220214010276
6       "C51"   1.2269584736225978
7       "C26"   1.2203471578902352
8       "C61"   1.2190762100675552
9       "C19"   1.2080674589235236
10      "C14"   1.2074147198905056
11      "C63"   1.206750047819268
12      "C52"   1.2051430846019162
13      "C34"   1.1960932075802668
14      "C59"   1.186922711687243
15      "C56"   1.1841688404382151
16      "C35"   1.1837484780107108
17      "C15"   1.182451898551215
18      "C29"   1.1821913019831896
19      "C12"   1.1800993388722651
20      "C55"   1.1755257049670735
21      "C24"   1.175037589853663
22      "C17"   1.1676369870653622
23      "C16"   1.1588507034574764
24      "C39"   1.1475132728957917
25      "C47"   1.14612193885565
26      "C22"   1.1340134845563254
27      "C101"  1.1309067391403302
28      "C13"   1.1203658777932923
29      "C46"   1.119112793710933
30      "C38"   1.1173039921849235
31      "C41"   1.11006151888716
32      "C40"   1.1061285013485165
33      "C18"   1.1052430073574848
34      "C45"   1.103890752144628
35      "C37"   1.1034434409399259
36      "C65"   1.1018134545003677
37      "C57"   1.1004882057221215
38      "C36"   1.0958282811902105
39      "C31"   1.0943889053459661
40      "C43"   1.094279903873983
41      "C10"   1.0936190954242895
42      "C66"   1.0935899101521043
43      "C28"   1.085988792445158
44      "C32"   1.0845407480432159
45      "C44"   1.077618594281165
46      "C49"   1.0775341436490882
47      "C48"   1.0726565224848388
48      "C42"   1.0643876605531166
49      "C23"   1.0614424296335876
50      "C33"   1.0555699446620956
51      "C58"   1.050609109559689
52      "C20"   1.0504654861397411
53      "C113"  1.0461070026034618
54      "C62"   1.0447622124238727
55      "C98"   1.040630483348009
56      "C68"   1.0331664619736538
57      "C105"  1.0321136466012524
58      "C27"   1.0312946098104023
59      "C86"   1.0301343609824418
60      "C92"   1.029297759021717
61      "C30"   1.02883008931889
62      "C69"   1.017739584620091
63      "C127"  1.0173974790741993
64      "C114"  1.016027543932544
65      "C82"   1.013651690012841
66      "C53"   1.0107860881772848
67      "C124"  1.0065077908416147
68      "C94"   0.9987223306578578
69      "C97"   0.9970201633770771
70      "C64"   0.9953957458452458
71      "C96"   0.9953046898125949
72      "C111"  0.9843195772194393
73      "C126"  0.9841042932870333
74      "C128"  0.9803232497592262
75      "C90"   0.9772709783527275
76      "C87"   0.9757621925074781
77      "C25"   0.972694161917751
78      "C88"   0.9723272981150329
79      "C103"  0.9706867284281467
80      "C125"  0.9667008689224187
81      "C01"   0.9657091867078631
82      "C95"   0.9580879953822644
83      "C67"   0.957937805004983
84      "C110"  0.9574757518387493
85      "C85"   0.9541623988424536
86      "C104"  0.9509179348993253
87      "C99"   0.9488985373983424
88      "C89"   0.9467397799776766
89      "C83"   0.9451541011596529
90      "C09"   0.9389405196765955
91      "C81"   0.9297857135310638
92      "C122"  0.9273912941259649
93      "C107"  0.9270645491610486
94      "C115"  0.9255790594836256
95      "C102"  0.9196914991385376
96      "C72"   0.917570198503481
97      "C50"   0.9160096091266053
98      "C108"  0.9150850552327898
99      "C79"   0.909509231033356
100     "C91"   0.9053582079598089
101     "C84"   0.8958702296865297
102     "C109"  0.8761377544007346
103     "C106"  0.8671373586094492
104     "C70"   0.8644722880290338
105     "C112"  0.8568413904544915
106     "C80"   0.8540853663893501
107     "C93"   0.8432195616244242
108     "C116"  0.8405878125873606
109     "C119"  0.8351134232054357
110     "C08"   0.8282797572116257
111     "C07"   0.8252295579725029
112     "C123"  0.8216658812447498
113     "C02"   0.8141785631288354
114     "C118"  0.7953127915092747
115     "C04"   0.7922908507387288
116     "C121"  0.7494933271150035
117     "C03"   0.7453159572652599
118     "C74"   0.7213651251811808
119     "C06"   0.7092608821171026
120     "C71"   0.6817503422504274
121     "C78"   0.6810065009509227
122     "C117"  0.6802089951307128
123     "C05"   0.6753799676843679
124     "C75"   0.6646188665720576
125     "C73"   0.664522740721276
126     "C76"   0.6515001369508422
127     "C77"   0.6437625284391406
128     "C120"  0.636479016610417


The general principle behind the calculations is that we run through the races in chronological order; and, for each race, we use the average points of the involved pipsqueaks in all past races to calculate a strength for the race. This strength value is, in turn, used to correct the points for each pipsqueak in the current race. The corrected points are included in the averages for each pipsqueak, and then we proceed to the next race. Below is a reasonably detailed (but not 100% complete) explanation of the algorithm if you are curious:




The original data source were historical database dumps gotten from Zak, which I converted to a list of races, in which each race is a "list" of associations (technically, a map) between pipsqueaks and their respective final classifications (for Z128: Duplode -> 1; BJ -> 2; Friker -> 3; etc.). In the explanation below, a step means picking the next race from the list and processing it into the accumulated results for the previous races.

At the end of each step, we have the following accumulated results:

  • average points per race for every pipsqueak in all races he/she took part thus far;
  • the average raw strength of the races thus far (what is a raw strength will be explained shortly);
  • the normalized strengths of every race thus far (the values in the plot above are the normalized strengths as of Z128); and
  • how many races were included thus far.
In each step, the next race is included in these results (accumulated over all previous races). That is done in the following way:

  • First, we perform a weighed sum of the average-points-per-race for all pipsqueaks who took part in the race. The square root of this weighed sum is the raw strength of the race.

    • Taking the square root of the sum was a somewhat arbitrary decision, that aims to avoid having 10x differences between strengths which, given the uses I'm thinking of, seems unreasonable.
    • The weighs make the sum biased towards the midfield (see Overdrijf's post, or my second message in the thread, for a rationale. Furthermore, It appears that biasing towards top pipsqueaks make the differences between races too small.
  • The newly-calculated raw strength is used to update the average of raw strengths.
  • The normalized strengths are the raw strengths divided by the updated average of raw strengths, so that a value of 1 corresponds to the average strength of a ZakStunts race (that can be easily seen in the plot).

    • These normalized strengths are calculated in an indirect way, using the previous and updated values of average raw strengths and the number of races thus far to normalize the current strengths and correct the values of previous races.
  • Next, we calculate the points for each race in the current race and multiply them by the newly-calculated normalized strength for the race (using it as a correction factor for the scores, in other words).

    • The scoring system used here is a modified version of ZakStunts scores (12, 11, etc.), which assigns points to all pipsqueaks below 12th place using the formula points=1/(position-11).
  • Finally, these corrected scores are used to update the average-points-per-race for every pipsqueak who took part in the race. That concludes the step; and so we can proceed to the next race.
Title: Re: Race strength estimation
Post by: zaqrack on April 22, 2012, 02:18:24 AM
The graph seems spot on to me, so I assume the formula must be working :)
Should three next steep be to use these race strengths to weight the results of the pipsqueaks and thus form an ultimate eternal rank list?
Title: Re: Race strength estimation
Post by: Overdrijf on April 22, 2012, 02:20:11 PM
Well, that certainly explains why I did so well on ZCT93...

(And on 117, and on 112, and basically in every race. Damn, I'm the king of the weak races. Pick your battles, they said.)
Title: Re: Race strength estimation
Post by: Duplode on April 22, 2012, 04:06:16 PM
Quote from: zaqrack on April 22, 2012, 02:18:24 AM
The graph seems spot on to me, so I assume the formula must be working :)
Should three next steep be to use these race strengths to weight the results of the pipsqueaks and thus form an ultimate eternal rank list?

Pretty much that - just add very big quotation marks on the "eternal"  :) There are so many assumptions to be made when choosing how to rank pipsqueaks...
Title: Re: Race strength estimation
Post by: Friker on April 23, 2012, 03:38:02 AM
eh? seems interesting.. let me think about it :)
Title: Re: Race strength estimation
Post by: Friker on April 23, 2012, 12:07:49 PM
Quote from: Friker on April 23, 2012, 03:38:02 AM
eh? seems interesting.. let me think about it :)

your system is "well-constructed" :) but still i would like to see some output from my thoughts. maybe in august ill write something :)
Title: Re: Race strength estimation
Post by: BonzaiJoe on April 24, 2012, 12:39:03 PM
I'm very impressed with the accuracy of that graph.

Average results are a problematic indicator because of the concept of listfillers, though. I would not like for my result on ZCT79 to be taken as an indication of my racing level, for example...

But by the looks of it, this problem doesn't undermine the algorithm.
Title: Re: Race strength estimation
Post by: Duplode on April 24, 2012, 04:51:39 PM
Quote from: BonzaiJoe on April 24, 2012, 12:39:03 PM
Average results are a problematic indicator because of the concept of listfillers, though. I would not like for my result on ZCT79 to be taken as an indication of my racing level, for example...

But by the looks of it, this problem doesn't undermine the algorithm.

To me it seems that listfillers should be somehow accounted for when calculating race strengths. If a top pipsqueaks spends a season sending only listfillers, his/hers participation ideally should not contribute as much for race strengths during that season. For rankings of pipsqueaks, though, it does seem fair to have a way to discount listfillers, though I have not settled on one thus far.

The real problem with these running average algorithms is related to handling different time scales. Should early races of a pipsqueak which improved slowly but steadily into a top pipsqueak weigh down the averages forever? One might think of dropping early races (say, by only including the past 60 races into the average) - but then, how do we deal with cases like Roy/Reiger?

(The above considerations made me realize that calculating dropping "extreme" results - the worst 25%, for instance - for each pipsqueak could solve some of these problems...)
Title: Re: Race strength estimation
Post by: Friker on April 24, 2012, 07:43:47 PM
Quote from: Duplode on April 24, 2012, 04:51:39 PM
Quote from: BonzaiJoe on April 24, 2012, 12:39:03 PM
Average results are a problematic indicator because of the concept of listfillers, though. I would not like for my result on ZCT79 to be taken as an indication of my racing level, for example...

But by the looks of it, this problem doesn't undermine the algorithm.

To me it seems that listfillers should be somehow accounted for when calculating race strengths. If a top pipsqueaks spends a season sending only listfillers, his/hers participation ideally should not contribute as much for race strengths during that season. For rankings of pipsqueaks, though, it does seem fair to have a way to discount listfillers, though I have not settled on one thus far.

The real problem with these running average algorithms is related to handling different time scales. Should early races of a pipsqueak which improved slowly but steadily into a top pipsqueak weigh down the averages forever? One might think of dropping early races (say, by only including the past 60 races into the average) - but then, how do we deal with cases like Roy/Reiger?

(The above considerations made me realize that calculating "extreme" results - the worst 25%, for instance - for each pipsqueak could solve some of these problems...)

This is another think I was thinkin about. Average is a very bad indicator. It should be a median or average from top 60 per cents of races.
Title: Re: Race strength estimation
Post by: CTG on February 01, 2014, 12:28:47 PM
It would be nice to see ZCT 129-149 in the stats. I think some of the ZSC2013 races were pretty hard.
Title: Re: Race strength estimation
Post by: alanrotoi on February 01, 2014, 09:12:03 PM
Is it possible to calculate zakstunts pipsqueaks performance per season?

For example an average between imaginary fastest lap and average of all final results.

It may say something about general perfomance including 1st place but for lack of stats we should do the same calculation between 1st place time and average 2nd-last place time.
Title: Re: Race strength estimation
Post by: CTG on February 06, 2015, 03:17:47 PM
Quote from: CTG on February 01, 2014, 12:28:47 PM
It would be nice to see ZCT 129-149 in the stats. I think some of the ZSC2013 races were pretty hard.

Duplode: could you please update those stats?
Title: Re: Race strength estimation
Post by: Duplode on February 07, 2015, 06:38:53 AM
Quote from: CTG on February 06, 2015, 03:17:47 PM
Duplode: could you please update those stats?

Here they are! Fortunately it wasn't difficult to get the (clunky, messy, undocumented) code from three years ago to work again. Plot:

(http://i49.photobucket.com/albums/f283/Duplode/rse-2014.png)

Z126-Z161:

(http://i49.photobucket.com/albums/f283/Duplode/rse-recent.png)

Data, sorted chronologically...


1       "C01"   0.9689400477956864
2       "C02"   0.8169024658050857
3       "C03"   0.7478094743173973
4       "C04"   0.7949415262372359
5       "C05"   0.677639508017131
6       "C06"   0.7116337738910322
7       "C07"   0.8279904326789597
8       "C08"   0.8310508366154873
9       "C09"   0.9420818239435098
10      "C10"   1.0972778898408067
11      "C11"   1.2404899424930709
12      "C12"   1.1840474601972015
13      "C13"   1.124114155813747
14      "C14"   1.2114542271138526
15      "C15"   1.1864078905618902
16      "C16"   1.162727735690297
17      "C17"   1.171543414546985
18      "C18"   1.1089406905463877
19      "C19"   1.212109149940057
20      "C20"   1.0539799065366733
21      "C21"   1.247335334941021
22      "C22"   1.137807421789967
23      "C23"   1.0649935743157235
24      "C24"   1.1789687766726773
25      "C25"   0.9759483918260397
26      "C26"   1.2244299316696852
27      "C27"   1.0347448924325184
28      "C28"   1.0896220590430292
29      "C29"   1.186146422146186
30      "C30"   1.0322721266809785
31      "C31"   1.0980502752261478
32      "C32"   1.0881691700870795
33      "C33"   1.0591014424531762
34      "C34"   1.1968395184366045
35      "C35"   1.186675827050426
36      "C36"   1.0811583543707717
37      "C37"   1.077402305685727
38      "C38"   1.0920956472475025
39      "C39"   1.1355196539355568
40      "C40"   1.1009690283447056
41      "C41"   1.1069528673472575
42      "C42"   1.0586761961920326
43      "C43"   1.0881245289825938
44      "C44"   1.079502836318097
45      "C45"   1.1056138374370517
46      "C46"   1.123911480968403
47      "C47"   1.1487808697986865
48      "C48"   1.073530365396353
49      "C49"   1.0698461700981488
50      "C50"   0.9170253239754842
51      "C51"   1.2256388760218293
52      "C52"   1.2084359855101359
53      "C53"   1.0138805448345958
54      "C54"   1.230100830983952
55      "C55"   1.1811679701291025
56      "C56"   1.181138013114387
57      "C57"   1.095312753011811
58      "C58"   1.0410976347846719
59      "C59"   1.1889970978565068
60      "C60"   1.246705158723104
61      "C61"   1.2186523436514123
62      "C62"   1.0404809789236207
63      "C63"   1.2115048611023715
64      "C64"   0.998089447533628
65      "C65"   1.1043051801041592
66      "C66"   1.096916991872554
67      "C67"   0.958986680861299
68      "C68"   1.0337562866527779
69      "C69"   1.0087598620715927
70      "C70"   0.8650927921523183
71      "C71"   0.6838412095390317
72      "C72"   0.920230735989811
73      "C73"   0.6666372759742343
74      "C74"   0.7238444443202098
75      "C75"   0.6667446862628902
76      "C76"   0.6537072386584194
77      "C77"   0.6460955451900426
78      "C78"   0.6835887235118633
79      "C79"   0.9127183957000091
80      "C80"   0.8571497164717864
81      "C81"   0.9330118799582122
82      "C82"   1.0171968221057057
83      "C83"   0.9484466516560793
84      "C84"   0.8989854425929414
85      "C85"   0.9575127217390671
86      "C86"   1.033437523165465
87      "C87"   0.979245234382722
88      "C88"   0.9758955778420796
89      "C89"   0.9500002906190882
90      "C90"   0.9808312155955641
91      "C91"   0.90913697352392
92      "C92"   1.0336759602936316
93      "C93"   0.84678025173352
94      "C94"   1.00262565160693
95      "C95"   0.9616690341956202
96      "C96"   0.9988325365932506
97      "C97"   1.0004956921389636
98      "C98"   1.0443681022662894
99      "C99"   0.9522585176695119
100     "C100"  1.3864666915197195
101     "C101"  1.1344522542473525
102     "C102"  0.9229873062738267
103     "C103"  0.973934107144398
104     "C104"  0.9543003756122189
105     "C105"  1.0356259904068026
106     "C106"  0.8704116651381018
107     "C107"  0.9305455620003085
108     "C108"  0.9185732330463009
109     "C109"  0.8794617945694458
110     "C110"  0.9609050588121039
111     "C111"  0.9879179564228385
112     "C112"  0.8598870905243342
113     "C113"  1.049866032397705
114     "C114"  1.0198398085440483
115     "C115"  0.9288924939867218
116     "C116"  0.843910754800388
117     "C117"  0.6829431422227926
118     "C118"  0.7983255980605544
119     "C119"  0.8382726824222874
120     "C120"  0.6388754109549367
121     "C121"  0.7522345128087627
122     "C122"  0.9307755960280949
123     "C123"  0.8249698996825673
124     "C124"  1.0100723350771101
125     "C125"  0.9703784608556298
126     "C126"  0.9878406384791253
127     "C127"  1.0210024728292988
128     "C128"  1.0137533770991995
129     "C129"  0.9691319709413175
130     "C130"  0.8183069076896105
131     "C131"  0.960763406601733
132     "C132"  0.9561603424161117
133     "C133"  1.0682626021949784
134     "C134"  1.100843257219523
135     "C135"  1.0643773901560338
136     "C136"  0.8539884609442169
137     "C137"  0.9128046380265993
138     "C138"  1.1718845649163696
139     "C139"  1.07745641609288
140     "C140"  1.0582642884803737
141     "C141"  0.9688228344946141
142     "C142"  0.9564228675082869
143     "C143"  0.8956600329934347
144     "C144"  0.9702091411932694
145     "C145"  1.0715087582028076
146     "C146"  0.9864259959232726
147     "C147"  0.9639372970562176
148     "C148"  0.925672351676703
149     "C149"  0.9690298358171286
150     "C150"  1.0973689693676674
151     "C151"  1.0346525275472949
152     "C152"  1.131985734713242
153     "C153"  1.1141132736088906
154     "C154"  1.0153874087851067
155     "C155"  0.931052503803616
156     "C156"  0.9796519979983589
157     "C157"  1.0443901571811245
158     "C158"  1.0487149142639867
159     "C159"  0.9478919544339005
160     "C160"  0.8522174320964528
161     "C161"  0.8349361390430942


...and by strength:


1       "C100"  1.3864666915197195
2       "C21"   1.247335334941021
3       "C60"   1.246705158723104
4       "C11"   1.2404899424930709
5       "C54"   1.230100830983952
6       "C51"   1.2256388760218293
7       "C26"   1.2244299316696852
8       "C61"   1.2186523436514123
9       "C19"   1.212109149940057
10      "C63"   1.2115048611023715
11      "C14"   1.2114542271138526
12      "C52"   1.2084359855101359
13      "C34"   1.1968395184366045
14      "C59"   1.1889970978565068
15      "C35"   1.186675827050426
16      "C15"   1.1864078905618902
17      "C29"   1.186146422146186
18      "C12"   1.1840474601972015
19      "C55"   1.1811679701291025
20      "C56"   1.181138013114387
21      "C24"   1.1789687766726773
22      "C138"  1.1718845649163696
23      "C17"   1.171543414546985
24      "C16"   1.162727735690297
25      "C47"   1.1487808697986865
26      "C22"   1.137807421789967
27      "C39"   1.1355196539355568
28      "C101"  1.1344522542473525
29      "C152"  1.131985734713242
30      "C13"   1.124114155813747
31      "C46"   1.123911480968403
32      "C153"  1.1141132736088906
33      "C18"   1.1089406905463877
34      "C41"   1.1069528673472575
35      "C45"   1.1056138374370517
36      "C65"   1.1043051801041592
37      "C40"   1.1009690283447056
38      "C134"  1.100843257219523
39      "C31"   1.0980502752261478
40      "C150"  1.0973689693676674
41      "C10"   1.0972778898408067
42      "C66"   1.096916991872554
43      "C57"   1.095312753011811
44      "C38"   1.0920956472475025
45      "C28"   1.0896220590430292
46      "C32"   1.0881691700870795
47      "C43"   1.0881245289825938
48      "C36"   1.0811583543707717
49      "C44"   1.079502836318097
50      "C139"  1.07745641609288
51      "C37"   1.077402305685727
52      "C48"   1.073530365396353
53      "C145"  1.0715087582028076
54      "C49"   1.0698461700981488
55      "C133"  1.0682626021949784
56      "C23"   1.0649935743157235
57      "C135"  1.0643773901560338
58      "C33"   1.0591014424531762
59      "C42"   1.0586761961920326
60      "C140"  1.0582642884803737
61      "C20"   1.0539799065366733
62      "C113"  1.049866032397705
63      "C158"  1.0487149142639867
64      "C157"  1.0443901571811245
65      "C98"   1.0443681022662894
66      "C58"   1.0410976347846719
67      "C62"   1.0404809789236207
68      "C105"  1.0356259904068026
69      "C27"   1.0347448924325184
70      "C151"  1.0346525275472949
71      "C68"   1.0337562866527779
72      "C92"   1.0336759602936316
73      "C86"   1.033437523165465
74      "C30"   1.0322721266809785
75      "C127"  1.0210024728292988
76      "C114"  1.0198398085440483
77      "C82"   1.0171968221057057
78      "C154"  1.0153874087851067
79      "C53"   1.0138805448345958
80      "C128"  1.0137533770991995
81      "C124"  1.0100723350771101
82      "C69"   1.0087598620715927
83      "C94"   1.00262565160693
84      "C97"   1.0004956921389636
85      "C96"   0.9988325365932506
86      "C64"   0.998089447533628
87      "C111"  0.9879179564228385
88      "C126"  0.9878406384791253
89      "C146"  0.9864259959232726
90      "C90"   0.9808312155955641
91      "C156"  0.9796519979983589
92      "C87"   0.979245234382722
93      "C25"   0.9759483918260397
94      "C88"   0.9758955778420796
95      "C103"  0.973934107144398
96      "C125"  0.9703784608556298
97      "C144"  0.9702091411932694
98      "C129"  0.9691319709413175
99      "C149"  0.9690298358171286
100     "C01"   0.9689400477956864
101     "C141"  0.9688228344946141
102     "C147"  0.9639372970562176
103     "C95"   0.9616690341956202
104     "C110"  0.9609050588121039
105     "C131"  0.960763406601733
106     "C67"   0.958986680861299
107     "C85"   0.9575127217390671
108     "C142"  0.9564228675082869
109     "C132"  0.9561603424161117
110     "C104"  0.9543003756122189
111     "C99"   0.9522585176695119
112     "C89"   0.9500002906190882
113     "C83"   0.9484466516560793
114     "C159"  0.9478919544339005
115     "C09"   0.9420818239435098
116     "C81"   0.9330118799582122
117     "C155"  0.931052503803616
118     "C122"  0.9307755960280949
119     "C107"  0.9305455620003085
120     "C115"  0.9288924939867218
121     "C148"  0.925672351676703
122     "C102"  0.9229873062738267
123     "C72"   0.920230735989811
124     "C108"  0.9185732330463009
125     "C50"   0.9170253239754842
126     "C137"  0.9128046380265993
127     "C79"   0.9127183957000091
128     "C91"   0.90913697352392
129     "C84"   0.8989854425929414
130     "C143"  0.8956600329934347
131     "C109"  0.8794617945694458
132     "C106"  0.8704116651381018
133     "C70"   0.8650927921523183
134     "C112"  0.8598870905243342
135     "C80"   0.8571497164717864
136     "C136"  0.8539884609442169
137     "C160"  0.8522174320964528
138     "C93"   0.84678025173352
139     "C116"  0.843910754800388
140     "C119"  0.8382726824222874
141     "C161"  0.8349361390430942
142     "C08"   0.8310508366154873
143     "C07"   0.8279904326789597
144     "C123"  0.8249698996825673
145     "C130"  0.8183069076896105
146     "C02"   0.8169024658050857
147     "C118"  0.7983255980605544
148     "C04"   0.7949415262372359
149     "C121"  0.7522345128087627
150     "C03"   0.7478094743173973
151     "C74"   0.7238444443202098
152     "C06"   0.7116337738910322
153     "C71"   0.6838412095390317
154     "C78"   0.6835887235118633
155     "C117"  0.6829431422227926
156     "C05"   0.677639508017131
157     "C75"   0.6667446862628902
158     "C73"   0.6666372759742343
159     "C76"   0.6537072386584194
160     "C77"   0.6460955451900426
161     "C120"  0.6388754109549367
Title: Re: Race strength estimation
Post by: CTG on February 07, 2015, 12:42:24 PM
Quote from: Duplode on February 07, 2015, 06:38:53 AM
Quote from: CTG on February 06, 2015, 03:17:47 PM
Duplode: could you please update those stats?

Here they are! Fortunately it wasn't difficult to get the (clunky, messy, undocumented) code from three years ago to work again.

Thank you very much! Great work!
Title: Re: Race strength estimation
Post by: zaqrack on February 09, 2015, 09:34:57 AM
We had a great season start, the usual summer wind-down and a miserable season finish. Thanks to CTG's tantrum and my move back from China for the last one.  >:(
Title: Re: Race strength estimation
Post by: zaqrack on February 09, 2015, 09:46:38 AM
added some context :)
Title: Re: Race strength estimation
Post by: CTG on February 09, 2015, 01:26:08 PM
Quote from: zaqrack on February 09, 2015, 09:34:57 AM
Thanks to CTG's tantrum and my move back from China for the last one.  >:(

Look on the bright side: at least it improved the stats of Akoss Poo. :D

However, my own participation and activity had also an effect on the tendency. It will be published from home.