Race strength estimation

[Duplode]

Question: suppose we had a consistent SWR-style metric for estimating pipsqueak strengths at a given instant. In such a case, which would be the best way of deciding if one race was stronger than the other? If you vote for 2 or 3, please also tell what would count as "top" or "midfield".

(The motivation for that is that I'm doing some speculative work on rankings, and want to validate some assumptions. Expect news about that in the next few days...)

[BonzaiJoe]

I think the most obvious would be to ask "how much work (*skill*luck) is required to attain nth position in this race", for all n, but the weight diminishing along some curve, perhaps linearly, perhaps with a bias for podium spots (because they are discriminately valued by pipsqueaks).

[Duplode]

Let us continue the discussion... I feel that perhaps a good case can be made for the counter-intuitive strength-of-midfield option: while even in weaker races there are two or three top pipsqueaks fighting for the victory,  if you looked at a scoreboard and saw Ayrton in 6th place you'd instantly know that was an exceptionally hard race. (By the way, by midfield I mean, roughly, the region around 2/5 of the way down the scoreboard.)

[zaqrack]

yes, I would agree with that. Another factor to consider could be the gap between the average finishing time of the three podiumers against the average or median finishing time of the top 12

Or to unleash the power of the accumulated history date (unified data is available from 2004, so already more than 100 tracks!
- gap between the podiumers compared to historic averages of the same value
- the position of some "key" pipsqueaks on the scoreboard against their historic average position (which is shown on the profile page)

[zaqrack]

also worth to mention, that any of the above factors could only work after a race is finished or near the final days.
For a similar estimate in the first two weeks of a race we could rely on the number of changes on the leading position and the number of participants sending replays.

[Chulk]

if you looked at a scoreboard and saw Ayrton in 6th place you'd instantly know that was an exceptionally hard race.

This is not entirely true... I think I can achieve a 5th place regularly if I had enough time to race at Zakstunts. That doesn't necessarily mean that every race where I had not enough time/motivation was specially hard. On the other hand, my average position might be 10th because I lack the time to race. Then maybe I just have a quiet month at work/university, raced a lot and finish 5th. Would you say that race was specially easy?.

I think there's no accurate way to measure something that has so many variables like time, motivation, amount of drivers, driver's car skill, teammates (of course it's easier to have a top driver as a teammate than a newbie), even the moment of the year has an impact on race results.

If I had to do it, I would see how much can time be improved in a given moment. For example, I don't think Renato Biker's legendary Countach replay cannot be improved as much as any of Roy's winning replays.

Other possibility is comparing winning time against imaginary sum of fastest sections, but that could also be really innacurate in races like the one Alan Rotoi wo with Ferrari, 1st race of 2005 I think...

[Friker]

after thinking about it i came to conclusion that quality of a race has no correlation with quality of pipsqueaks. if you have to use the quality of pipsqueaks it's probably a number of top pipsqueaks actively participating. but i think it's better to measure it as "how hard was to take nth place". (probably first seven)

[Duplode]

Starting with Friker's: the problem is how to convert "how hard it was" into a number. I would have to disagree with you in that strength of involved pipsqueaks does have correlation with that, although of course it is far from a perfect one. Ideally, the measure would include how strong the replays were (see Chulk's reply); but for that one would need to either 1. have a perfect (!?) reference lap for every track (!!); or 2. have section data for all races and then rely on imaginary sum of fastest sections (and then be subject to distortions like Z47, as Chulk points out; not to mention multi-car ZakStunts makes this method significantly more unreliable). Done consistently, both alternatives would mean massive amounts of work for data collection; and, for that reason, I am more inclined to, at least as a first approximation, find ways of measuring race strength which do not require watching or driving replays.

The Chulk-with-a-listfiller v. Chulk-in-full-power problem is a hard one, too. If a pipsqueak drives most races below his/hers full potential, in most races it would be appropriate to use a strength-with-listfillers value. But how to deal with the races driven at full power? And how to distinguish these from a fluke result? (There might be a way that alleviates this problem; maybe factoring in both a ranking counting all races by a pipsqueak and another including only his/hers best x% results.)

Zak's suggestions could be useful, and fun to play with. The "how hard it is" approach of Friker and BJ leads to some interesting, and possibly feasible, considerations on computing strengths for a specific position in a race instead of for the race as a whole.

Later I will post a sketch of what the calculation scheme I am planning looks like ATM. And finally: it is all just numbers, I know. But numbers can be amusing, too 

[Friker]

hmm.. ok some of my thoughts: (i did not write them earlier because some seemed to be off-topic)
-ordering should be symmetric and transitive ("a<b implies b[not less then]a" and "a<b and b<c implies a<c") (the third rule is irreflexivity but it's not as important in this situation)
-there is no unbiased way how to tell pipsqueak A is better then pipsqueak B: a day before yesterday i read some article where was used term "dominate" and it means "A dominates B when A is quicker then B in all cases" (quicker because they were comparing the speed of algorithms). ok, this is nice idea but algorithms always run as fast as they can. pipsqueaks do not. even if we want to substract worst places for each pipsqueak, this is not correct because of some pipsqueaks as zak. but still the substraction is a good heuristic. probably a few first races should not count to that domination's calculation. (some kind of newbie status)
-another way how to calculate something is: take the 5th (or 6th) place and calculate percentual value of other places. (for example 1st time was 60sec 5th time was 90sec - a value of 1st time is -33.33% which could be converted to -33.33pts) 5th place because its somewhere middle between 1 and 10 which are only relevant times. places below 10 would not be counted.
ok that was about A is better than B. what about race R is stronger then race S?
-hm? and what does it means - race R is stronger then race S? this is far more complex problem then it seems to be. i give you my off-topic opinion about races:
[more off-topic]
i play (frisbee) ultimate. we train, we play, we have tourneys. also we compete a lot. but a good turney means there is fun at a field and also at a party. And also a good turney has to have a good breakfast. the fun is more important than the final place. so a strength of a race is one thing but the fun around is something else.
[/more off-topic]
-i still think that BJ's "how hard is to take nth place" is still the most accurate formulation of what the strong race is.
-another way could be  through "what is the sum of being dominated of pipsqueaks in top N" where N should be 6-10 (really good races will have this number 0, still good races will have somewhere around 4-10, with bigger N there is more accent to worse pipsqueaks)
-the straightforward way is to count how many good pipsqueaks were participating in the race - here is a place to take that percentual thing comes into play

there are many other approaches described and probably a way better than this one but we don't know about them.. (so what we are doing here is probably reinviting a wheel)

[Duplode]

P.S.: Your comments about Ultimate made me realized I still had not mentioned one important point about what I have in mind. Ultimately, measuring race strength is not an end in itself. Rather, the goal is using race strengths to weigh the points earned in the races when calculating, for instance, average score rankings for pipsqueaks. For those of you who remember Mark's SWR, the general idea somewhat resembles its "quality points", but calculated in a way that makes better use of the available historical data.

[CTG]

I think it's useless to define the strength of a race - it depends on too many factors. Now telling a single one: personal motivation levels. When BJ finishes 9th, you just can't take it seriously. And how can you decide that Ayrton would be able to drive 2, 5 or 10 seconds better than the winning time?

As for me, you mustn't calculate the strength of the race - you should rather feel it.

[Duplode]

Valid points, but you don't need to read too much into it. All I want with these ideas about "race strength" is to have a factor which accounts for  the intuition that , e.g. your victory in Z49 should have more weight than the one in Z73. It doesn't have to be definitive or perfectly accurate. In fact, it is not even necessary that such race strength numbers compose a meaningful ranking on their own - as long as the factors are reasonable, the relative order of the races is of little consequence.

(To make it a bit clearer: suppose we adopted the current F1 system - 25, 18, 12... -  for base scores. If the "strength factor" was 1.1 for Z49 and 0.8 for Z73, your victories in them would translate into 25*1.1 = 27.5 and 25*0.8 = 20 points, respectively.)

[CTG]

Valid points, but you don't need to read too much into it. All I want with these ideas about "race strength" is to have a factor which accounts for  the intuition that , e.g. your victory in Z49 should have more weight than the one in Z73. It doesn't have to be definitive or perfectly accurate. In fact, it is not even necessary that such race strength numbers compose a meaningful ranking on their own - as long as the factors are reasonable, the relative order of the races is of little consequence.

(To make it a bit clearer: suppose we adopted the current F1 system - 25, 18, 12... -  for base scores. If the "strength factor" was 1.1 for Z49 and 0.8 for Z73, your victories in them would translate into 25*1.1 = 27.5 and 25*0.8 = 20 points, respectively.)

No problem with the math part, it will be a great work if you create this system. But I think it's a little bit useless. Maybe the problem is with me: I can't think on Stunts racing as a major part of my life - not like in the past 10 years (LOL, I'll be "10 years old" on Friday). Nowadays I drive only when I have time and mood.

[zaqrack]

Nowadays I drive only when I have time and mood.

So you have finally also achieved the state of Stunts-Zen

[Duplode]

No problem with the math part, it will be a great work if you create this system. But I think it's a little bit useless. Maybe the problem is with me: I can't think on Stunts racing as a major part of my life

No worries - in a way, you're right. What makes a ranking useful, anyway? Only if the positions are worth anything (as in SWR-ISM), or else if you are going to brag about your scores (which I surely won't ). To me, these plans are both a programming exercise about [suppressed long-winded explanations about the tricks I am trying to learn/apply]  and a way of playing with a fun question that roams around the community since time immemorial.

LOL, I'll be "10 years old" on Friday

Hey, I wasn't aware your "birthday" is in April too