Stunts Forum

Stunts - the Game => Stunts Chat => Topic started by: Duplode on January 28, 2023, 07:24:25 AM

Title: Elo-like ratings for ZakStunts: The Folyami Project
Post by: Duplode on January 28, 2023, 07:24:25 AM
I'm delighted to finally release the ZakStunts Folyami ratings, an Elo-like rating system for ZakStunts! You can check the ratings right now at the Southern Cross site (https://scr.stunts.hu/folyami.html). Below is a quick Q&A about the ratings -- if you have extra questions, feel free to ask them!

Why are there two rankings?

Given that Folyami ratings are pretty dynamic, responding rather quickly to changes in form/current performance, it felt appropriate to have something the more permanent to go along with the ranking of current ratings. That being so, there is also a historical ranking, which lists the highest ever rating reached by pipsqueaks.

Why am I not showing up in the rankings?

There are basically two possibilities:


So don't worry: no one gets excluded from the rankings, and you just have to keep racing for (re)joining it  :)

How do the ratings work?

Here are links to a summary of the ZakStunts-specific aspects of the rankings (https://scr.stunts.hu/folyami.html#about-the-ratings) and a technical overview of the rating system (https://scr.stunts.hu/folyami/system.html). (I have attached the latter here as a PDF, in case you find that easier to read.)

Can other competitions be included in the ratings?

The Folyami project began as an offshoot of my earlier investigations about race strength (https://forum.stunts.hu/index.php?topic=3635.0), which is why the ratings came into being as ZakStunts-only. Ideally, it would be nice to follow illustrious predecessors such as WRL and SWR and make Folyami an omni-rating covering all Stunts competitions. While I do want to explore ways of achieving that at some point, it can't help but be a project for the long term. Not only there are decades of competition results to be reviewed and formatted, but also harmonising the competitions into a single system could prove challenging, especially given how much the currently active competitions differ from each other.

Will there be an update of the race strength ranking?

Sure! I will add race strengths to the site as soon as I figure out a few details about how to best present the data. By the way, if you have any suggestions of additional features and visualisations for the ratings and the historical data thereof, please do let me know!
Title: Re: The Folyami Project: Elo-like ratings for ZakStunts
Post by: Argammon on January 28, 2023, 07:38:12 AM
Wow, impressive!  :)

I did not read how the system works,so please allow me a stupid question:

The current ratings make sense to me,.but I am surprised about the historical ones. Stunts champions of the past like Roy, Bonsai Joe,.and Alain have surprisingly low all-time best ratings.

Could there be an issue with rating inflation?

Edit: It would be really cool if there was a seperate rating for each car. For example, is it true that I underperform with IMSA cars or is that just a myth? But I guess we don't have enough data for any reliable single-car ratings

Title: Re: The Folyami Project: Elo-like ratings for ZakStunts
Post by: Duplode on January 28, 2023, 01:52:47 PM
Quote from: Argammon on January 28, 2023, 07:38:12 AMThe current ratings make sense to me,.but I am surprised about the historical ones. Stunts champions of the past like Roy, Bonsai Joe,.and Alain have surprisingly low all-time best ratings.

Could there be an issue with rating inflation?

Excellent question! The system probably underrates a bit results from 2001 and 2002, simply because there was too little time the ratings to settle. Besides that, I think the difference is mainly that, in later years, longer periods of dominance, earlier champions being defeated on-track by upstarts, and comebacks at the highest level (see e.g. Renato Biker) have become more common. Here are two charts to better illustrate the trends. The first one shows the ratings of current ranking leaders:

current-lead.png

The second shows the mean ratings at each race including pipsqueaks who were active in the previous 12 rounds and reached the 5-race initial cutoff:

mean-rating-12-5.png

Quote from: Argammon on January 28, 2023, 07:38:12 AMEdit: It would be really cool if there was a seperate rating for each car. For example, is it true that I underperform with IMSA cars or is that just a myth? But I guess we don't have enough data for any reliable single-car ratings

Though the results wouldn't be very reliable indeed, that would be a fun thing to try! One detail is that we'd only have to decide if races like Z82 (https://zak.stunts.hu/tracks/ZCT082) or Z98 (https://zak.stunts.hu/tracks/ZCT098) should count as IMSA races.
Title: Re: The Folyami Project: Elo-like ratings for ZakStunts
Post by: Argammon on January 28, 2023, 02:18:19 PM
 not unproblematic, but the results could change Ayrton's GTO rating, BJ's Jaguar rating etc
Title: Re: Elo-like ratings for ZakStunts: The Folyami Project
Post by: Cas on January 28, 2023, 06:32:31 PM
Great work!  As always, super professional!
Title: Re: Elo-like ratings for ZakStunts: The Folyami Project
Post by: KyLiE on January 29, 2023, 05:20:07 AM
Excellent work!  I really appreciate the effort that you put into this.  I'll be sure to keep an eye on my rating in the future! :)
Title: Re: Elo-like ratings for ZakStunts: The Folyami Project
Post by: Duplode on January 29, 2023, 01:10:42 PM
Thanks @Cas and @KyLiE ! On keeping an eye on your rating and tracking your progress, right now the only feature (if I can call it that!) the page has related to that is the handmade "personal best" announcement in the header. If you have suggestions on what could be added to make it easier to follow, I'm all ears!
Title: Re: Elo-like ratings for ZakStunts: The Folyami Project
Post by: Argammon on January 29, 2023, 03:48:49 PM
Quote from: Duplode on January 29, 2023, 01:10:42 PMThanks @Cas and @KyLiE ! On keeping an eye on your rating and tracking your progress, right now the only feature (if I can call it that!) the page has related to that is the handmade "personal best" announcement in the header. If you have suggestions on what could be added to make it easier to follow, I'm all ears!

There could be a rating graph for each pipsqueak. Yeah, I love creating work for you.  :o
Title: Re: Elo-like ratings for ZakStunts: The Folyami Project
Post by: Overdrijf on January 29, 2023, 04:34:03 PM
Very impressive. I would almost encourage you to publish it in some scientific journal for statistics.
Title: Re: Elo-like ratings for ZakStunts: The Folyami Project
Post by: Cas on January 29, 2023, 07:47:20 PM
"30 year old cars game becomes world famous after a scientific paper is published on tournament statistics"  8)
Title: Re: Elo-like ratings for ZakStunts: The Folyami Project
Post by: Overdrijf on January 29, 2023, 08:03:26 PM
No seriously, applied to something like real car races, or horse races, or, well, you get the point, lots of races. Maybe athletics events, open water swimming, sailing. Preferably stuff where a record time doesn't tell enough of a story because there's a different track every time, different weather, the field of competitors keeps changing. Which second rate cycling teams get access to the Tour de France this year? Is Verstappen more dominant than Schumacher was? Which kayakers do we call for the Ötz Trophy? I am sure there are already lots of systems for tracking the performance of these things over time, and I have no idea how they compare to this version. But it is one of those things for which there oddly enough doesn't seem to be a single good standard.

The Folyami system seems remarkably well thought out, almost too good to be worth it for a few dozen people community player a game. That Youtube account 'd better be bringing in tons if new people, if we have a new powerful tool like this to rank them with.
Title: Re: Elo-like ratings for ZakStunts: The Folyami Project
Post by: Cas on January 29, 2023, 11:23:56 PM
This rank system is worth a video and/or wiki article, by the way
Title: Re: Elo-like ratings for ZakStunts: The Folyami Project
Post by: Duplode on January 30, 2023, 12:34:23 AM
@Argammon A history graph for each pipsqueak could be done; it's something I'll consider doing in the mid term. The main thing to deal with will be that I will surely want to generate the 90-odd pages/graphs automatically, which means I will have to a little bit of integration between the program that generates the rankings and the program that generates the Southern Cross pages.

@Overdrijf Good point about variability of conditions. I think I could use both kind of examples: variable conditions, to see how the ratings fare, and stable conditions, to test the underlying hypothesis of the gamma performance model. Also, when I go looking for test cases, it is probably better not to focus much on motorsport, as it tends to involve too many confounding factors (for the ratings, there's car and team differences; for the performance model, there's changing car behaviour due to fuel load and tyre wear).

(There's also the matter of finding metrics that express how good the ratings actually are. Though I have done a little bit of that using NDCG (https://forum.stunts.hu/index.php?msg=77619), it would be probably sensible to dig deeper into the topic.)
Title: Re: Elo-like ratings for ZakStunts: The Folyami Project
Post by: Overdrijf on February 12, 2023, 04:28:05 PM
I'm not sure if it would be a bit silly here, but one of the best bits about elo systems is that little dopamine hit from winning a match and gaining 30 elo. Would that be something that could work here, like a collumn for last month or for how much you went up since last month?
Title: Re: Elo-like ratings for ZakStunts: The Folyami Project
Post by: alanrotoi on February 12, 2023, 04:40:18 PM
Quote from: Duplode on February 12, 2023, 04:07:40 PMThe ratings (https://scr.stunts.hu/folyami.html) have been updated for ZCT258! This is the first public update of the rankings.

Cool! Where is the last month list? It would be cool to keep every month list too, isn't it? It's the basement of more statistics  :)
Title: Re: Elo-like ratings for ZakStunts: The Folyami Project
Post by: Duplode on February 12, 2023, 11:45:49 PM
Quote from: Overdrijf on February 12, 2023, 04:28:05 PMI'm not sure if it would be a bit silly here, but one of the best bits about elo systems is that little dopamine hit from winning a match and gaining 30 elo. Would that be something that could work here, like a collumn for last month or for how much you went up since last month?

While I'm being deliberate with adding comparison features to the page, I agree some way of quickly having a sense of recent progress is desirable, and the most basic way of adding that is through the rating change from the previous round. So yes, I'll add this one  :) (Rating changes rather than ranking position changes, as the latter might not be all that meaningful given how often people leave and rejoin the current ranking.)

Quote from: alanrotoi on February 12, 2023, 04:40:18 PMCool! Where is the last month list? It would be cool to keep every month list too, isn't it? It's the basement of more statistics   :)

This might take a little longer, as I'll have to soup up the generation of pages for the mini-site. But I'll work on it as well! (Doing that will also help with Argammon's suggestion of pipsqueak-wise data/charts.)
Title: Re: Elo-like ratings for ZakStunts: The Folyami Project
Post by: Duplode on February 14, 2023, 07:49:57 PM
@Overdrijf The rating change from the previous round is now shown with the current ranking.
Title: Re: Elo-like ratings for ZakStunts: The Folyami Project
Post by: Overdrijf on February 14, 2023, 08:42:48 PM
Haha, I'm on my way up! I overtook a person! No 7th place against many talented drivers driving great laps can stop my advance!

... Yes, this is working for me. Thanks!
Title: Re: Elo-like ratings for ZakStunts: The Folyami Project
Post by: alanrotoi on March 08, 2023, 05:06:21 AM
I may sound hard or angry but it is just a bad sleepy english, don't get it wrong please :-*

I want to discuss a case: juank_23 vs Eddie Brother.
They raced in the same track only twice and both Eddie was beaten.
Juank pipsqueak in 4 tracks and the positions were:
11/14
9/12
12/17
13/15

And Eddie raced in 5 tracks:
14/17
14/15
10/12
10/12
10/16

How in the hell Eddie has 1419 points and Juank_23 1415? I mean I understand there is some kind of calculations but from my guts I know it is a bit unfair for juank.

I repeat, I'm a big fan of this project so I want to see it as precise as possible.


And about exponential distribution, do I understand correctly if I say when you win more thsn once in a row you get less and less points? You know how hard is to win or keep a podium position for more than few races. It should be rewarded. Maybe I get it wrong. I see the numbers and the large explanation with calculations and graphs but I just can say how many point I'll get/loose if I finish in x position in the next track. Maybe the page needs also a less technic explanation and a point simulator.
Title: Re: Elo-like ratings for ZakStunts: The Folyami Project
Post by: Overdrijf on March 08, 2023, 08:10:57 AM
In this type of rankings you get points for beating people with a good position on the board compared to you. If you win a race you beat everyone, so you gain points and the average other pipsqueak loses points. Do if you beat all the same people again the next point you get less points. Let's say the first time Duplode was 100 points above you, CTG was 50 points above you and I was 200 points below you, then on the second race Duplode was only 75 points above you, CTG 25 points above you and I 210 points below you. That's a less impressive victory, in an elo-like rating.

It also makes sense because the main point of these ratings is not to reward you for winning, but to end up ranking you well compared to the others. As you close in on your rightful position the changes should get smaller and smaller, not bigger, or you'd just end up with a bazillion points and the tanking would be useless.



As for the case of Eddie and Juank, apparently Eddie had some impressive rankings compared to drivers with good point totals. Maybe there were some.extra good drivers driving that month driving the midfield closer to his position or something? Actually, I should study how this ranking works better bwfore trying to answer rhis one. It is just one example though, out of lots of years of Zakstunts.
Title: Re: Elo-like ratings for ZakStunts: The Folyami Project
Post by: alanrotoi on March 08, 2023, 12:37:49 PM
Quote from: Overdrijf on March 08, 2023, 08:10:57 AMIn this type of rankings you get points for beating people with a good position on the board compared to you. If you win a race you beat everyone, so you gain points and the average other pipsqueak loses points. Do if you beat all the same people again the next point you get less points. Let's say the first time Duplode was 100 points above you, CTG was 50 points above you and I was 200 points below you, then on the second race Duplode was only 75 points above you, CTG 25 points above you and I 210 points below you. That's a less impressive victory, in an elo-like rating.

It also makes sense because the main point of these ratings is not to reward you for winning, but to end up ranking you well compared to the others. As you close in on your rightful position the changes should get smaller and smaller, not bigger, or you'd just end up with a bazillion points and the tanking would be useless.


So maybe is better way to win a race then second and then win again that to win 2 in a row? :o
Title: Re: Elo-like ratings for ZakStunts: The Folyami Project
Post by: Overdrijf on March 08, 2023, 03:56:20 PM
Quote from: alanrotoi on March 08, 2023, 12:37:49 PMSo maybe is better way to win a race then second and then win again that to win 2 in a row? :o

Win-lose-win will probably net you a higher current rating than win-win-lose, yes, but lose-win-win might get you even higher, and a long record full of win streaks and loss streaks should give you the same average and a higher top than the same number of wins and losses without streaks.

(That's what I think happens anyway.)
Title: Re: Elo-like ratings for ZakStunts: The Folyami Project
Post by: Duplode on March 09, 2023, 04:53:05 AM
Those are good questions, @alanrotoi ! The comments by @Overdrijf make for a good outline of the issues (the basic ideas of the Elo system do apply, in spite of the substantial changes Folaymi incorporates); I'll try to help with the details.

Let's begin with Eddie and Juank. Here are their full rating histories:

+-----++--------------------+
|     ||      Eddie Brother |
+=====++====================+
| C14 || 1429.7218603558094 |
+-----++--------------------+
|  P1 || 1417.3501392006976 |
+-----++--------------------+
| C15 || 1366.7000422242472 |
+-----++--------------------+
| C16 ||  1434.054239919096 |
+-----++--------------------+
|  P3 || 1419.3351433565706 |
+-----++--------------------+

+-----++--------------------+
|     ||           Juank_23 |
+=====++====================+
|  C9 || 1498.4609688295145 |
+-----++--------------------+
| C10 || 1447.4398384088088 |
+-----++--------------------+
| C11 || 1453.4838164837072 |
+-----++--------------------+
| C14 ||  1450.633155577332 |
+-----++--------------------+
| C15 || 1415.8621307766819 |
+-----++--------------------+

Both of them have exactly five counting races, the minimum needed to enter the rankings (before that, the ratings are calculated and accounted for, but not shown in the rankings as they are too volatile and unreliable). That being so, both show up on the historical ranking with their final rating, from their only entry in the ranking ever, which is a pretty unusual situation. Eddie is only ahead of Juank due to Eddie having a strong (relatively to his own performances) result at C16 (10/16, ahead of Usrin and Ben Snel), Juank having a weak (again, relatively to his own performances) result at C15 (14/16). Juank was still ahead at C15, by about 50 points, but wasn't around to bounce back in the following races (also note that the provisional factors make the ratings more volatile than usual over the first few races of a pipsqueak). In summary, Juank had really bad luck with the timing of what became his historical rating, and we'd probably see a different picture had the two of them raced a few rounds more than the bare minimum needed to enter the rankings.

Quote from: alanrotoi on March 08, 2023, 05:06:21 AMdo I understand correctly if I say when you win more thsn once in a row you get less and less points?

Basically yes. Repeated wins will make the rating difference between you and your opponents larger (more positive), so your expected score (i.e. the winning probability given by the model) will be larger (that's what the exponents in the formulas for EEloX and WX in the expected score section do), which in turn reduces the points gained from the matches. The reduction is pretty gradual, though. Marco's winning streak in late 2016 and early 2017 gives a decent illustration of the effect:

+------++--------------------+
| C181 || 1912.5228198049917 |
+------++--------------------+
| C182 || 1962.9674878144833 |
+------++--------------------+
| C183 ||  1998.219045817359 |
+------++--------------------+
| C184 || 2029.4221198139737 |
+------++--------------------+
| C185 || 2059.1942707075377 |
+------++--------------------+
| C187 ||  2082.898975327112 |
+------++--------------------+

Total gains per race were: +50 (C182), +36 (C183), +33 (C184), +30 (C185), +23 (C187). In summary, the idea is that surprising results should change the ratings faster than unsurprising ones, so that the ratings reflect how performances evolve, and repeated results get less and less surprising. It is also important to note, as Overdrijf points out, that a lot depends on what the field of opponents was like. For a somewhat extreme illustration, we can look at my ratings in the early 2018 races:

+------++--------------------+
| C197 || 2064.9343648958343 |
+------++--------------------+
| C198 || 2069.9753551755457 |
+------++--------------------+
| C199 ||   2073.89726641674 |
+------++--------------------+
| C200 ||  2080.930573062527 |
+------++--------------------+
| C201 || 2087.8455341750982 |
+------++--------------------+
| C202 || 2067.5222038971915 |
+------++--------------------+

C198 (+5) and C199 (+4) had some of the smallest fields in ZakStunts history, which helps explaining how my wins there gave me less points than my second places in C200 (+7) and C201 (+7).

Quote from: alanrotoi on March 08, 2023, 12:37:49 PMSo maybe is better way to win a race then second and then win again that to win 2 in a row? :o

Assuming there's a loss after the two consecutive wins, it will likely be a bit better. Here are rating changes for a few scenarios -- to simplify things, for these calculations I'll use a pure Elo system (without the extra Folyami modifiers) with K = 18 and just two pipsqueaks starting from equal ratings:


It goes just as Overdijf predicted. For individual races, there's somewhat of a "the higher they are, the harder they fall" effect, which is part of the reason why there are a few measures to limit a bit the effect of unrepresentative bad results. Still, those differences are expected to even out over a longer series of races.

Quote from: alanrotoi on March 08, 2023, 05:06:21 AMI see the numbers and the large explanation with calculations and graphs but I just can say how many point I'll get/loose if I finish in x position in the next track. Maybe the page needs also a less technic explanation and a point simulator.

There's a little bit of that, at least as far as getting a sense of how large the values are, in the first table at the end of the article (https://scr.stunts.hu/folyami/system.html#points-gained-after-a-win). For instance, let's suppose you have 2200 rating, and Renato 2300. If you win and Renato ends in fourth, you will gain 9.7 points from him (row "-100" for the rating difference, column "3" for the position difference). Keep in mind that the 9.7 is just for your match with Renato; the full picture requires considering the matches against everyone in the scoreboard.

I like the simulator idea, by the way! Later I'll look into at least adding at least a JavaScript calculator for 1v1 matches  :)
Title: Re: Elo-like ratings for ZakStunts: The Folyami Project
Post by: Overdrijf on March 12, 2023, 09:44:04 AM
Quote from: Duplode on March 12, 2023, 09:08:13 AMThe ratings (https://scr.stunts.hu/folyami.html) have been updated for ZCT259!

It looks like the site itself hasn't actually updated yet...
Title: Re: Elo-like ratings for ZakStunts: The Folyami Project
Post by: Daniel3D on March 12, 2023, 11:42:08 AM
After reading some bit about the history of Elo. I realised that when I was a member of the chess club the club ranking was also an Eli rating. I never realised that because any ranking would have placed me in the same position. Since in the two years I was there I have solved every chess problem that was presented but I did not win any matches. So with 0 wins, i had a solid last place. Since I knew and expected to be last i never looked twice at the scoreboard or point system.
Title: Re: Elo-like ratings for ZakStunts: The Folyami Project
Post by: Duplode on March 12, 2023, 02:33:35 PM
@Overdrijf If it's still not loading the updated tables, try Ctrl + Shift + R to bypass the cache. (I wonder if I should adjust anything on the site with respect to that.)

@Daniel3D I'm now wondering what a chess problem leaderboard could be like... chess.com seems to have such a thing (https://www.chess.com/leaderboard/tactics), but I have no idea what the success percentage is meant to be. (To my eyes, timed competitive problem solving doesn't look like it would be an enjoyable pastime, but I'm not actually a chess player, so who knows.)
Title: Re: Elo-like ratings for ZakStunts: The Folyami Project
Post by: Daniel3D on March 12, 2023, 05:00:50 PM
The chess problems were for learning, they had no leaderboard. But I had no problems with them. I just couldn't think ahead well enough to win a match. Teacher's syndrome they called it.
Title: Re: Elo-like ratings for ZakStunts: The Folyami Project
Post by: Cas on March 13, 2023, 01:04:46 AM
When I play chess, I've noticed I am much better if I've played many times against that person. I usually suck the first time I play with a certain player. It looks like my game intuition is heavily based on reaction and anticipation to other player's game style instead of pure strategy.

I remember I used to have an Elo rating a long time ago, when I played chess over Yahoo. I don't remember my numbers, though.
Title: Re: Elo-like ratings for ZakStunts: The Folyami Project
Post by: Erik Barros on July 05, 2023, 01:09:32 AM
I need to save this image before my graph starts going down hehehe
Title: Re: Elo-like ratings for ZakStunts: The Folyami Project
Post by: Argammon on August 31, 2023, 09:19:31 AM
I have the feeling there is rating inflation going on. I haven't done any thorough analysis but here are some indications:


-If you sum up the plusses and minuses the end result is positive every month. This month the sum is +54.
-After C263 I am slightly in front of Duplode. Then I win C264 and Duplode wins C265, yet he suddenly is in front of me.
-Alain who I had close battles with is 200 rating points below me. I think I improved a little compared to the years 2003-2006 but probably not so tremendously.

This is a great project Duplode. It Would be nice if you could look into the issue more thoroughly then I did.  :)
Title: Re: Elo-like ratings for ZakStunts: The Folyami Project
Post by: Duplode on September 01, 2023, 04:32:15 AM
Quote from: Argammon on August 31, 2023, 09:19:31 AMIt Would be nice if you could look into the issue more thoroughly then I did.  :)

Sure! Later I will rerun some diagnostics involving global properties of the ratings. These can be tricky to interpret given how the pool of pipsqueaks is always changing, but they might provide some signal on whether there's something unusual going on. Meanwhile, here's my preliminary take on the things you've noted above:

On the monthly sum of changes: The ranking shown in the site doesn't give a full picture, as active pipsqueaks that still haven't reached five races aren't included. Once they are accounted for, the sum of changes in ZCT265 goes down from +54 to +32. For the preceding couple of races, the effect is larger still: +81 to -214 in ZCT264 (the sum now dominated by first entries, which I'm regarding as rating changes from the 1500 default value), and +73 to -96 in ZCT263. This kind of fluctuation stems from the provisional factors, which mean the changes involving pipsqueaks who have recently entered the ranking won't add up to zero, and they don't necessarily lead to meaningful inflation or deflation in the long run.

On our relative positions: You had reached the top of the ranking in ZCT263 by a single point (2192 to 2191). Your win in ZCT264 opened a bit of distance (2213 to 2196), which I have now clawed back (2219 to 2220 -- a single point again, but now in may favour). After one win each, we're almost where we were started (not exactly at the same place, as the gap opening up slightly in ZCT264 led to a slightly larger swing in the opposite direction in ZCT265.)

On Alan: Given how the current rating tends to reflect recent form, looking a bit further back to Alan's personal best reached in 2022 (ZCT254, 2213) might provide a fuller picture. It's also worth noting that Alan lost more than 100 points in ZCT262 alone, even if, a few races later, the overall effect looks less dramatic: given that Alan's rating has remained broadly stable since, it appears likely that, assuming a similar form were to be maintained, his rating would have converged to ~2040 even without the ZCT262 outlier, though that would perhaps happen closer to the end of the season. (And of course, a win or two by Alan would have him narrowing the gap quite a lot.)
Title: Re: Elo-like ratings for ZakStunts: The Folyami Project
Post by: Cas on February 15, 2024, 10:05:30 PM
How far back can this be done?  Is it possible to have a condensed graph of the whole 21st century?  (That is, beginning with the opening of ZakStunts in 2001)
Title: Re: Elo-like ratings for ZakStunts: The Folyami Project
Post by: Duplode on February 16, 2024, 12:25:41 AM
Quote from: Cas on February 15, 2024, 10:05:30 PMHow far back can this be done?  Is it possible to have a condensed graph of the whole 21st century?  (That is, beginning with the opening of ZakStunts in 2001)

The data does go all the way back to 2001. That graph might become a little busy, but if eloratings.net (https://www.eloratings.net/graph) can do it, why can't we? :)
Title: Re: Elo-like ratings for ZakStunts: The Folyami Project
Post by: Cas on February 16, 2024, 07:25:32 PM
Yep, probably it'd be better if it were interactive and you can enable or disable pipsqueaks or groups of pipsqueaks and zoom in the time.... Or... it could just be a very high resolution image that we can zoom in and out offline.
Title: Re: Elo-like ratings for ZakStunts: The Folyami Project
Post by: Duplode on June 22, 2024, 02:01:30 AM
Quote from: Duplode on February 16, 2024, 12:25:41 AM
Quote from: Cas on February 15, 2024, 10:05:30 PMHow far back can this be done?  Is it possible to have a condensed graph of the whole 21st century?  (That is, beginning with the opening of ZakStunts in 2001)

The data does go all the way back to 2001. That graph might become a little busy, but if eloratings.net (https://www.eloratings.net/graph) can do it, why can't we? :)

All right, here is the full history graph (https://scr.stunts.hu/files/outrascoisas/folyami-graph.html)! I'll eventually tidy it up a bit and add to the site proper.

This uses the same chart library than the aforementioned eloratings.net, and so the controls are the same: hover to show the legend and highlight individual pipsqueaks, click to the left of the vertical axis to clear highlights, drag horizontally to zoom, double-click to reset zoom, drag the lower right corner to resize. All races up to ZCT274 are covered, with ZCTP1, ZCTP2 and ZCTP3 being displayed as 14.5, 15.5 and 16.5 for the sake of expediency.

(Cc @Cas , @Argammon , @alanrotoi and everyone else curious about seeing more of the historical data  :))

2025-05-08 edit: The interactive graph has since been integrated with the site, with monthly updates. It is now available on this page (https://scr.stunts.hu/folyami/graph.html).
Title: Re: Elo-like ratings for ZakStunts: The Folyami Project
Post by: Frieshansen on June 22, 2024, 09:31:10 AM
Great work! Very impressive to see the whole history in one big interactive chart.
Title: Re: Elo-like ratings for ZakStunts: The Folyami Project
Post by: Duplode on November 15, 2024, 06:42:13 PM
Quote from: Argammon on August 31, 2023, 09:19:31 AMI have the feeling there is rating inflation going on.

Quote from: Duplode on September 01, 2023, 04:32:15 AM
Quote from: Argammon on August 31, 2023, 09:19:31 AMIt Would be nice if you could look into the issue more thoroughly then I did.  :)

Sure! Later I will rerun some diagnostics involving global properties of the ratings. These can be tricky to interpret given how the pool of pipsqueaks is always changing, but they might provide some signal on whether there's something unusual going on.

I've finally managed to get back to this. Inflation can be a tricky thing to diagnose, as there are so many ways to slice the data, so here are a few different ways to look at the global evolution of the ratings.

The most intuitive thing to check, perhaps, is the mean rating at each race of everyone in the current ranking (which covers those pipsqueaks with a included race entry over the previous four rounds):

(https://i.imgur.com/fMfWZVu.png)

While closer analysis could be merited, at first glance there aren't obvious signs of systemic inflation or deflation. The long-term shifts that do exist can be attributed to the evolution of the competition as a whole:


Another way of looking at the evolution is focusing on a specific position in the current ranking. For instance, below is the rating of the sixth-placed pipsqueak (starting from 2003 so that the unrepresentative low values from the first few races don't mess up the chart):

(https://i.imgur.com/aIouMGt.png)

This gives us a different perspective, which has more directly to do with the depth of the field. Still, there is no continued growth of the sixth-place rating, which typically oscillated around 1900 (up to the Middle Ages) or 1850 (in the current era).

Looking at the current ranking probably gives us a perspective better attuned to the action on the racetrack at any given moment. What about the overall pool of pipsqueaks, though? Below is the evolution mean rating of all pipsqueaks, including those that haven't made it to the ranking due to having fewer than five included races:

(https://i.imgur.com/YIOskFo.png)

Rating changes in the Folyami system don't add up to zero because of provisional status, which makes ratings change faster for pipsqueaks with less than twelve races. The direction of the global change depends on which share of those newcomers succeed in getting strong results and improving their ratings. That being so, a slow decrease of the global mean is expected.

Lastly, for something of a compromise between looking at the current ranking mean and the global mean, we might try looking at the mean rating across pipsqueaks that have made it to historical ranking (that is, which have at least five included races and thus made it at least once to the current ranking):

(https://i.imgur.com/SZ0lLeu.png)

That looks remarkably stable, with the mean having remained pretty much static throughout the Middle Ages, and seeing only a very mild downward shift in the last few seasons.
Title: Re: Elo-like ratings for ZakStunts: The Folyami Project
Post by: Duplode on November 23, 2024, 12:58:00 PM
A small but useful update to the Folyami graph (https://scr.stunts.hu/folyami/graph.html): the legend is now sorted according to the current ranking. That helps with something @alanrotoi asked for a long time ago: a convenient way to check past rankings. Now you can hover the mouse over (or touch) any chosen race (to make that easier, use the handles under the chart to zoom into a smaller range), and the legend below will, in effect, display the ranking:

(https://i.imgur.com/kwT2xCL.png)
Title: Re: Elo-like ratings for ZakStunts: The Folyami Project
Post by: Duplode on May 08, 2025, 12:30:44 PM
Some Forum bookkeeping: I have split the monthly updates of the ratings into a separate thread on the ZakStunts area of the Forum (https://forum.stunts.hu/index.php?topic=4433.0); the general discussion of the rating system and the site features remains here on the original topic. By the way, the I plan to post the ZCT285 update tonight ;)

Edit: Of course I was going to do all that and then announce the split on the thread I didn't mean to  :D Now fixed.