Race strength estimation

[CTG]

No problem with the math part, it will be a great work if you create this system. But I think it's a little bit useless. Maybe the problem is with me: I can't think on Stunts racing as a major part of my life

No worries - in a way, you're right. What makes a ranking useful, anyway? Only if the positions are worth anything (as in SWR-ISM), or else if you are going to brag about your scores (which I surely won't ). To me, these plans are both a programming exercise about [suppressed long-winded explanations about the tricks I am trying to learn/apply]  and a way of playing with a fun question that roams around the community since time immemorial.

LOL, I'll be "10 years old" on Friday

Hey, I wasn't aware your "birthday" is in April too 

13th April 2002, as far as I remember. Damn, I was sooo young...

[Chulk]

13th April 2002, as far as I remember. Damn, I was sooo young...

Is this the day you joined the forum or your first Zakstunts replay?

BTW: HAPPY B-DAY!

[CTG]

HAHAHAHA thank you!

[Chulk]

HAHAHAHA thank you!

You didn't answer my question, mister...

[CTG]

Replay, ZCT 13.

[Chulk]

Then mine is somewhere around February 2005. Although that lap wasn't valid or something because I finished with 9:99.99... don't remember what happened. First VALID replay was in March, no idea about the day I sent it.

[alanrotoi]

When was created your first ghost? 

[CTG]

When was created your first ghost? 

Suck my Indy!

[Overdrijf]

I'd say the best metric is the strength of the drivers just below the podium, numbers 4-8 or so. Even if there are two or three guys fighting an awesome battle at the top you never know how the rest of the field turns out. If there is a strong subtop it doesn't really matter if the real top is battling extra hard, the race is a good one anyway.

[/ontopic, please do continue sucking his Indy]

[CTG]

What about time hiding? Imagine two possibilities.

Z437 (whatever)

1 Inavoeg 45.50
2 Roy Wiegerinck 45.55
3 Mark Nailwood 45.65

...but all of them sent their best during the quiet days, Ben Snel was leading for 20 days with 52.70.

Same track, paralel universe:

1 Inavoeg 46.00
2 Roy Wiegerinck 46.75
3 Mark Nailwood 47.20

Not as tight, not as fast as the former one, but the leading time board is quite long (7 different leaders, milliseconds war during the whole month) and only smaller improvements took place during the quiet days.

Which one is stronger and more entertaining?

[Duplode]

I'd say the best metric is the strength of the drivers just below the podium, numbers 4-8 or so. Even if there are two or three guys fighting an awesome battle at the top you never know how the rest of the field turns out. If there is a strong subtop it doesn't really matter if the real top is battling extra hard, the race is a good one anyway.

Nice way of putting that argument 

What about time hiding? Imagine two possibilities.

Z437 (whatever)

1 Inavoeg 45.50
2 Roy Wiegerinck 45.55
3 Mark Nailwood 45.65

...but all of them sent their best during the quiet days, Ben Snel was leading for 20 days with 52.70.

Same track, paralel universe:

1 Inavoeg 46.00
2 Roy Wiegerinck 46.75
3 Mark Nailwood 47.20

Not as tight, not as fast as the former one, but the leading time board is quite long (7 different leaders, milliseconds war during the whole month) and only smaller improvements took place during the quiet days.

Which one is stronger and more entertaining?

Well, if heavy time hiding demotivated midfielders and led to weak replays just below the podium I agree the race would be less "strong" in the limited sense I am talking about here  (to some extent, that applies to races with "impossible" sent very early in which no one bothers to challenge the leaders). This situation, too, suggests that calculations involving not only the final positions but also final times could improve things; as well as Friker/BJ's strength-per-position ideas. (Yesterday I rewrote the algorithms I am using in my tests in a simpler way; so now it is feasible to try adding some more sophisticated features to it.)

I will return to this topic tonight with some actual numbers, so that you can judge how (im)plausible this whole thing is 

[Duplode]

Okay, time to show some concrete stuff. What you see below is a plot of race strengths for every ZakStunts race up to Z128. The calculations were done in pretty much the way I conceived them about a month ago - later I will try to incorporate some of your suggestions to see what happens. As CTG and others pointed out, there is not that much meaning in these results per se, but still you can tell me how reasonable they appear to be.



(Clicking on the image will open a larger version.)
Here are the plotted strength values, first in chronological order and then sorted in descending order of strengths.

Code Select Expand


1       "C01"   0.9657091867078631
2       "C02"   0.8141785631288354
3       "C03"   0.7453159572652599
4       "C04"   0.7922908507387288
5       "C05"   0.6753799676843679
6       "C06"   0.7092608821171026
7       "C07"   0.8252295579725029
8       "C08"   0.8282797572116257
9       "C09"   0.9389405196765955
10      "C10"   1.0936190954242895
11      "C11"   1.2363536177595087
12      "C12"   1.1800993388722651
13      "C13"   1.1203658777932923
14      "C14"   1.2074147198905056
15      "C15"   1.182451898551215
16      "C16"   1.1588507034574764
17      "C17"   1.1676369870653622
18      "C18"   1.1052430073574848
19      "C19"   1.2080674589235236
20      "C20"   1.0504654861397411
21      "C21"   1.2431761847373568
22      "C22"   1.1340134845563254
23      "C23"   1.0614424296335876
24      "C24"   1.175037589853663
25      "C25"   0.972694161917751
26      "C26"   1.2203471578902352
27      "C27"   1.0312946098104023
28      "C28"   1.085988792445158
29      "C29"   1.1821913019831896
30      "C30"   1.02883008931889
31      "C31"   1.0943889053459661
32      "C32"   1.0845407480432159
33      "C33"   1.0555699446620956
34      "C34"   1.1960932075802668
35      "C35"   1.1837484780107108
36      "C36"   1.0958282811902105
37      "C37"   1.1034434409399259
38      "C38"   1.1173039921849235
39      "C39"   1.1475132728957917
40      "C40"   1.1061285013485165
41      "C41"   1.11006151888716
42      "C42"   1.0643876605531166
43      "C43"   1.094279903873983
44      "C44"   1.077618594281165
45      "C45"   1.103890752144628
46      "C46"   1.119112793710933
47      "C47"   1.14612193885565
48      "C48"   1.0726565224848388
49      "C49"   1.0775341436490882
50      "C50"   0.9160096091266053
51      "C51"   1.2269584736225978
52      "C52"   1.2051430846019162
53      "C53"   1.0107860881772848
54      "C54"   1.228220214010276
55      "C55"   1.1755257049670735
56      "C56"   1.1841688404382151
57      "C57"   1.1004882057221215
58      "C58"   1.050609109559689
59      "C59"   1.186922711687243
60      "C60"   1.243762215547041
61      "C61"   1.2190762100675552
62      "C62"   1.0447622124238727
63      "C63"   1.206750047819268
64      "C64"   0.9953957458452458
65      "C65"   1.1018134545003677
66      "C66"   1.0935899101521043
67      "C67"   0.957937805004983
68      "C68"   1.0331664619736538
69      "C69"   1.017739584620091
70      "C70"   0.8644722880290338
71      "C71"   0.6817503422504274
72      "C72"   0.917570198503481
73      "C73"   0.664522740721276
74      "C74"   0.7213651251811808
75      "C75"   0.6646188665720576
76      "C76"   0.6515001369508422
77      "C77"   0.6437625284391406
78      "C78"   0.6810065009509227
79      "C79"   0.909509231033356
80      "C80"   0.8540853663893501
81      "C81"   0.9297857135310638
82      "C82"   1.013651690012841
83      "C83"   0.9451541011596529
84      "C84"   0.8958702296865297
85      "C85"   0.9541623988424536
86      "C86"   1.0301343609824418
87      "C87"   0.9757621925074781
88      "C88"   0.9723272981150329
89      "C89"   0.9467397799776766
90      "C90"   0.9772709783527275
91      "C91"   0.9053582079598089
92      "C92"   1.029297759021717
93      "C93"   0.8432195616244242
94      "C94"   0.9987223306578578
95      "C95"   0.9580879953822644
96      "C96"   0.9953046898125949
97      "C97"   0.9970201633770771
98      "C98"   1.040630483348009
99      "C99"   0.9488985373983424
100     "C100"  1.3818428622781516
101     "C101"  1.1309067391403302
102     "C102"  0.9196914991385376
103     "C103"  0.9706867284281467
104     "C104"  0.9509179348993253
105     "C105"  1.0321136466012524
106     "C106"  0.8671373586094492
107     "C107"  0.9270645491610486
108     "C108"  0.9150850552327898
109     "C109"  0.8761377544007346
110     "C110"  0.9574757518387493
111     "C111"  0.9843195772194393
112     "C112"  0.8568413904544915
113     "C113"  1.0461070026034618
114     "C114"  1.016027543932544
115     "C115"  0.9255790594836256
116     "C116"  0.8405878125873606
117     "C117"  0.6802089951307128
118     "C118"  0.7953127915092747
119     "C119"  0.8351134232054357
120     "C120"  0.636479016610417
121     "C121"  0.7494933271150035
122     "C122"  0.9273912941259649
123     "C123"  0.8216658812447498
124     "C124"  1.0065077908416147
125     "C125"  0.9667008689224187
126     "C126"  0.9841042932870333
127     "C127"  1.0173974790741993
128     "C128"  0.9803232497592262


Code Select Expand


1       "C100"  1.3818428622781516
2       "C60"   1.243762215547041
3       "C21"   1.2431761847373568
4       "C11"   1.2363536177595087
5       "C54"   1.228220214010276
6       "C51"   1.2269584736225978
7       "C26"   1.2203471578902352
8       "C61"   1.2190762100675552
9       "C19"   1.2080674589235236
10      "C14"   1.2074147198905056
11      "C63"   1.206750047819268
12      "C52"   1.2051430846019162
13      "C34"   1.1960932075802668
14      "C59"   1.186922711687243
15      "C56"   1.1841688404382151
16      "C35"   1.1837484780107108
17      "C15"   1.182451898551215
18      "C29"   1.1821913019831896
19      "C12"   1.1800993388722651
20      "C55"   1.1755257049670735
21      "C24"   1.175037589853663
22      "C17"   1.1676369870653622
23      "C16"   1.1588507034574764
24      "C39"   1.1475132728957917
25      "C47"   1.14612193885565
26      "C22"   1.1340134845563254
27      "C101"  1.1309067391403302
28      "C13"   1.1203658777932923
29      "C46"   1.119112793710933
30      "C38"   1.1173039921849235
31      "C41"   1.11006151888716
32      "C40"   1.1061285013485165
33      "C18"   1.1052430073574848
34      "C45"   1.103890752144628
35      "C37"   1.1034434409399259
36      "C65"   1.1018134545003677
37      "C57"   1.1004882057221215
38      "C36"   1.0958282811902105
39      "C31"   1.0943889053459661
40      "C43"   1.094279903873983
41      "C10"   1.0936190954242895
42      "C66"   1.0935899101521043
43      "C28"   1.085988792445158
44      "C32"   1.0845407480432159
45      "C44"   1.077618594281165
46      "C49"   1.0775341436490882
47      "C48"   1.0726565224848388
48      "C42"   1.0643876605531166
49      "C23"   1.0614424296335876
50      "C33"   1.0555699446620956
51      "C58"   1.050609109559689
52      "C20"   1.0504654861397411
53      "C113"  1.0461070026034618
54      "C62"   1.0447622124238727
55      "C98"   1.040630483348009
56      "C68"   1.0331664619736538
57      "C105"  1.0321136466012524
58      "C27"   1.0312946098104023
59      "C86"   1.0301343609824418
60      "C92"   1.029297759021717
61      "C30"   1.02883008931889
62      "C69"   1.017739584620091
63      "C127"  1.0173974790741993
64      "C114"  1.016027543932544
65      "C82"   1.013651690012841
66      "C53"   1.0107860881772848
67      "C124"  1.0065077908416147
68      "C94"   0.9987223306578578
69      "C97"   0.9970201633770771
70      "C64"   0.9953957458452458
71      "C96"   0.9953046898125949
72      "C111"  0.9843195772194393
73      "C126"  0.9841042932870333
74      "C128"  0.9803232497592262
75      "C90"   0.9772709783527275
76      "C87"   0.9757621925074781
77      "C25"   0.972694161917751
78      "C88"   0.9723272981150329
79      "C103"  0.9706867284281467
80      "C125"  0.9667008689224187
81      "C01"   0.9657091867078631
82      "C95"   0.9580879953822644
83      "C67"   0.957937805004983
84      "C110"  0.9574757518387493
85      "C85"   0.9541623988424536
86      "C104"  0.9509179348993253
87      "C99"   0.9488985373983424
88      "C89"   0.9467397799776766
89      "C83"   0.9451541011596529
90      "C09"   0.9389405196765955
91      "C81"   0.9297857135310638
92      "C122"  0.9273912941259649
93      "C107"  0.9270645491610486
94      "C115"  0.9255790594836256
95      "C102"  0.9196914991385376
96      "C72"   0.917570198503481
97      "C50"   0.9160096091266053
98      "C108"  0.9150850552327898
99      "C79"   0.909509231033356
100     "C91"   0.9053582079598089
101     "C84"   0.8958702296865297
102     "C109"  0.8761377544007346
103     "C106"  0.8671373586094492
104     "C70"   0.8644722880290338
105     "C112"  0.8568413904544915
106     "C80"   0.8540853663893501
107     "C93"   0.8432195616244242
108     "C116"  0.8405878125873606
109     "C119"  0.8351134232054357
110     "C08"   0.8282797572116257
111     "C07"   0.8252295579725029
112     "C123"  0.8216658812447498
113     "C02"   0.8141785631288354
114     "C118"  0.7953127915092747
115     "C04"   0.7922908507387288
116     "C121"  0.7494933271150035
117     "C03"   0.7453159572652599
118     "C74"   0.7213651251811808
119     "C06"   0.7092608821171026
120     "C71"   0.6817503422504274
121     "C78"   0.6810065009509227
122     "C117"  0.6802089951307128
123     "C05"   0.6753799676843679
124     "C75"   0.6646188665720576
125     "C73"   0.664522740721276
126     "C76"   0.6515001369508422
127     "C77"   0.6437625284391406
128     "C120"  0.636479016610417


The general principle behind the calculations is that we run through the races in chronological order; and, for each race, we use the average points of the involved pipsqueaks in all past races to calculate a strength for the race. This strength value is, in turn, used to correct the points for each pipsqueak in the current race. The corrected points are included in the averages for each pipsqueak, and then we proceed to the next race. Below is a reasonably detailed (but not 100% complete) explanation of the algorithm if you are curious:



The original data source were historical database dumps gotten from Zak, which I converted to a list of races, in which each race is a "list" of associations (technically, a map) between pipsqueaks and their respective final classifications (for Z128: Duplode -> 1; BJ -> 2; Friker -> 3; etc.). In the explanation below, a step means picking the next race from the list and processing it into the accumulated results for the previous races.

At the end of each step, we have the following accumulated results:

• average points per race for every pipsqueak in all races he/she took part thus far;
• the average raw strength of the races thus far (what is a raw strength will be explained shortly);
• the normalized strengths of every race thus far (the values in the plot above are the normalized strengths as of Z128); and
• how many races were included thus far.

In each step, the next race is included in these results (accumulated over all previous races). That is done in the following way:

• First, we perform a weighed sum of the average-points-per-race for all pipsqueaks who took part in the race. The square root of this weighed sum is the raw strength of the race.
  
  
  
  • Taking the square root of the sum was a somewhat arbitrary decision, that aims to avoid having 10x differences between strengths which, given the uses I'm thinking of, seems unreasonable.
    
  • The weighs make the sum biased towards the midfield (see Overdrijf's post, or my second message in the thread, for a rationale. Furthermore, It appears that biasing towards top pipsqueaks make the differences between races too small.
• The newly-calculated raw strength is used to update the average of raw strengths.
• The normalized strengths are the raw strengths divided by the updated average of raw strengths, so that a value of 1 corresponds to the average strength of a ZakStunts race (that can be easily seen in the plot).
  
  
  
  • These normalized strengths are calculated in an indirect way, using the previous and updated values of average raw strengths and the number of races thus far to normalize the current strengths and correct the values of previous races.
• Next, we calculate the points for each race in the current race and multiply them by the newly-calculated normalized strength for the race (using it as a correction factor for the scores, in other words).
  
  
  
  • The scoring system used here is a modified version of ZakStunts scores (12, 11, etc.), which assigns points to all pipsqueaks below 12th place using the formula points=1/(position-11).
• Finally, these corrected scores are used to update the average-points-per-race for every pipsqueak who took part in the race. That concludes the step; and so we can proceed to the next race.

[zaqrack]

The graph seems spot on to me, so I assume the formula must be working
Should three next steep be to use these race strengths to weight the results of the pipsqueaks and thus form an ultimate eternal rank list?

[Overdrijf]

Well, that certainly explains why I did so well on ZCT93...

(And on 117, and on 112, and basically in every race. Damn, I'm the king of the weak races. Pick your battles, they said.)

[Duplode]

The graph seems spot on to me, so I assume the formula must be working
Should three next steep be to use these race strengths to weight the results of the pipsqueaks and thus form an ultimate eternal rank list?

Pretty much that - just add very big quotation marks on the "eternal"  There are so many assumptions to be made when choosing how to rank pipsqueaks...