Messages

1

Stunts Chat / Re: Stunts WIKI

« on: January 22, 2021, 11:39:09 PM »

Yes, probably DEFAULT.TRK, VANCOUVR.TRK and DEFAULT.RPL deserve their own articles. I have already told this, but the version of Stunts I first met was Brøderbund 1.0. It clearly had gone through other hands before so what I will say about it not necessarily describes features of the original 1.0 version, yet some items might.

On a related note, later I will add a track map of DEFAULT.TRK to its article, so that readers can find out whether the one in their copy has been modified, which indeed is a common occurrence. Loved the tidbit about the extra intro screen, by the way 

Which item are you referring to?  Is it VANCOUVR.TRK?

The one I was thinking of was the left corner bias -- the bias CTG has been warning us of through all those years. I have only realised the 1990 and 1991 versions differ on that last week, and it left me flabbergasted.

2

Stunts Chat / Re: Stunts WIKI

« on: January 22, 2021, 01:51:57 AM »

Hmm, is there a track (Vancouver) inside DEFAULT.RPL ? Mine (BB 1.1) is 2.8 KB, so it cannot contain both DEFAULT.TRK and VANCOUVR.TRK... 

In the 1.1 versions as originally distributed, there is a DEFAULT.RPL replay which was driven on the "hidden" VANCOUVR track. viniciusferrao and dstien have covered the details in this thread. (I probably should add some footnotes to the table to better explain this sort of thing.)

3

Stunts Chat / Game version comparison at the Wiki

« on: January 21, 2021, 11:30:59 PM »

I have added a work-in-progress set of tables summarising the differences between the game versions to the relevant Wiki article. If you see anything missing, please let us know and/or edit it in. Oh, and one of those items might surprise you! 

4

Stunts Chat / Re: Race strength estimation, revisited

« on: January 17, 2021, 03:54:44 PM »

I can't help by seeing a very marked change around C75. The whole "attitude" of the curve changes from there on.... The "Paleo-ZakStunts" and the "Neo-ZakStunts"   

I can think of a few factors that may cause that, the main one perhaps being the large fields typical of 2002-2006 races. Position prediction errors are more likely with more pipsqueaks involved, and my alternative strategy for computing the NDCG probably exacerbates that. It might be possible to offset that to some extent by further increasing the weights given to higher positions in the NDCG.

5

Stunts Chat / Re: Race strength estimation, revisited

« on: January 16, 2021, 11:51:39 PM »

When a measurement of a certain function of time f(t) against t looks this wild in a graph, my first thought is "maybe t is not my variable".

Though I haven't gotten to the qualitative part yet, your suggestion made me have a closer look at a few selected points in the graph. That ultimately led to a course correction, so thanks for getting me back on track  More specifically, I realised that:

• Due to a programming mistake, my NDCG code only accounted for newbies from their second race onward, messing up the calculations for races in which there were debuts. Fixing that removed a lot of artifacts from the results (I have replaced the charts in my previous post; if you look at them now you'll see the fluctuations got noticeably tamer).
• After the fix, doing comparative tests with varying Elo parameters has quickly shown the NDCG as specified in the paper is not as robust as we'd like it to be. The main problem is that predicting race results simply by putting ratings in order means we don't distinguish between a pipsqueak coming out ahead against a rating gap of, say, 1 point from doing the same against a 100 point gap. That being so, if ratings are close negligible fluctuations can have a disproportionate effect over the NDCG.

Those observations have rekindled my interest in using simulations to obtain the predicted results for the NDCG calculations, as a way of dealing with the robustness issue (for instance, the average positions of two pipsqueaks over a batch of simulations will be much closer if they have a 1 point gap than in the case of a 100 point gap). Here are some charts, without and with smoothing:





Quite a bit better. I don't think this alternative strategy is flawless, either (for one, it probably underrates predictions for races in which ratings are on the whole close to each other), but I'm far more confident about the results now.

6

Stunts Chat / Re: Race strength estimation, revisited

« on: January 16, 2021, 12:59:46 AM »

I have implemented the NDCG metric from the paper dreadnaut had suggested. (NDCG is one way to measure how close the results you might have predicted from the ratings before the race were to what actually happened, with a value of 1 meaning perfect agreement, and accuracy at higher positions on the scoreboard being given extra weight.)

As one might expect, the NDCG for ZakStunts races fluctuates wildly:



An exponential smoothing of the time series, though, shows that for the last several years of ZakStunts the fluctuations have been around 0.55, which is in the same ballpark of the results in the paper:



One thing I would like to achieve with NDCG thus far is using it to fine tune the parameters of my Elo engine. While parameter adjustments lead to localised changes in NDCG here and there, looking at the timeline as a whole the overall effect is very small. I have also attempted to use predicted positions obtained from simulations instead of just using the order of the ratings (I suspect that may ultimately give more informative NDCG results), but as far as the influence of Elo parameters is concerned that made no difference. To make progress on this front, I may also have to turn to the questions Cas has been raising here and try to think of a metric for how reasonable the evolution of individual pipsqueak ratings is.

7

Stunts Chat / Re: What IS Stunts anyway?

« on: January 10, 2021, 01:13:17 PM »

I think an important aspect of Stunts is how it handles grip, oversteer and understeer. It doesn't need to be super realistic (in fact, it's probably better if it isn't), but any Stunts-inspired physics engine should give you a clear sense of the grip limits and make it possible to get it wrong at a corner both by leaving the track "through the tangent" and by spinning, as well as to sometimes correct spins through countersteering. More generally, the handling feel of the cars (at least the non-cheat ones) should not be too arcade-like (in the "change direction however you want" sense) but also it shouldn't feel like you are always driving on an icy lake (a problem seen with some remakes and demos over the years).

For something unrealistic, I would nominate the possibility of changing direction in mid-air by taking jumps while sliding or spinning. In this case, I guess there is a fine line between fun and ludicrous for potential implementations.

A curious case is that of jump boosts, that is, the way accelerating through a jump can, depending on your choice of gear/RPM on takeoff, get you some extra speed upon landing. As important a part of the driving techniques repertory as it is, it is easy to forget that 1990 Stunts didn't have it at all! (And 1990 Stunts is no obscure version with limited distribution: Luke Loehrer created the Competition Car back in the day specifically for the benefit of its users, and it was the one I had back in the nineties.) I'm unsure about how essential jump boosts really are; good arguments can probably be drawn either way. There also is its partner-in-crime, the aerodynamic resistance bug which makes it possible for most cars (the Carrera is a rare exception) to stay above their flat track top speed without getting slowed down. Changing that would nerf jump boosts by making the very high speed ones temporary.

8

Stunts Forum & Portal / Re: Newbie guide & Driving school

« on: January 09, 2021, 11:15:51 PM »

I have added a section about the GAR scoreboard to the Newbie Guide, just after the one about replay visibility. Please let me know if you think I have forgotten anything there.

9

Competition 2021 / Re: Cars and rules for 2021

« on: January 09, 2021, 09:24:54 PM »

I like the 2021 rotation. While CTG's case against the Melange makes sense, I think the Audi is the most IMSA-like of the DTM cars, while the BMW is the least, so it should play out okay.

For 2021, I'd like to test single-race car bonuses: the points assigned by the track author will only apply for that race, and then be reverted. As such, author points don't need to sum up to zero anymore. Points rotation will continue as normal.

That's nice!

A new rule will require replays to be replayable from Options -> Load replay; this means participants will have to test powergear replays before uploading. If the car crashes, the replay will be invalid. I expect this rule to mostly affect extreme powergear laps, so it should not be a big obstacle.

While I can't help but feel this is a bit harsh, it seems very few laps would be affected by this rule. In fact, even my unstable Z232 replay passes this test (that Overdrijf replay is really one of a kind!). So I guess I can live with it, especially given the added bonus of not having to think too hard about validation...

10

Competition 2021 / Re: Cars and rules for 2021

« on: January 05, 2021, 08:26:06 PM »

Uhm... what Duplode says is making me think I should always include an original alternative in R4K when the race is based on a custom-made car. I'll implement that.

Well, I'd say that would be going too far in the other directions. I think the requirements for ZakStunts are a bit different: given how it tries to cater to as broad a range of tastes as possible (for instance, through multi-car races or the GAR scoreboard), it makes sense for it to go the extra mile when it comes to the ease of getting started. Besides that, highlighting new or overlooked custom cars is becoming part of R4K's identity already, and I don't see a need for diluting that.

(By the way, I don't think custom cars are a big barrier to entry, as long as it is easy to figure out where to get and how to install them. Maybe there is some useful data to be gathered from past scoreboards on whether custom cars have had any effect over newbie activity.)

But with more than 15 custom cars now available it feels as a limitation that only 5 are selected.
Maybe we can expand that to 9?
That way we have a selection of 20 car's instead of 16.
Or extended to 8 with a track bonus car (selected by track creator) so total is 20 as well..

The two main concerns I'd have with making more cars available would be with the balance of the bonus system, as Cas notes (more cars means more bonus points to be redistributed every race, and therefore wilder bonus swings), and with it taking too long for some cars to resurface and have a fair shot at a podium. Regardless of that, though, the track bonus car idea is quite interesting!

11

Competition 2021 / Re: Cars and rules for 2021

« on: January 05, 2021, 04:41:18 PM »

I think it is important to always make the eleven original cars available, not just as a matter of tradition but also of accessibility. If ZakStunts is an all-cars competition, it should be possible for a newbie who doesn't know anything about custom cars to be able to race with any car they choose from their mod-less Stunts installation.

12

Stunts Chat / Re: Race strength estimation, revisited

« on: January 05, 2021, 04:37:30 PM »

This may sound pretty crazy, but I'll extrapolate this in a purely philosophical way towards physics. Would it be accurate to call these extra degrees of freedom "internal"?  And if so, can we, in the same way that we ignore internal forces within a floating object, or in which charges inside a Faraday cage remain isolated from the environment, just leave these freedoms out?

Yup, that makes sense. A bare Elo rating abstracts away all those internal degrees of freedom. To put it in another way, if a strong pipsqueak only has time for five minute listfillers for a whole season, the other pipsqueaks will eventually come to expect the listfillers, regardless of what could be possible in more favourable conditions. Perhaps there is a way to split ratings into a long-term baseline and a more volatile part by somehow accounting by "internal" factors such as pipsqueak activity; it's territory that has yet to be charted.

13

Stunts Chat / Re: Race strength estimation, revisited

« on: January 03, 2021, 10:24:58 PM »

My thought was that they assumed single races. Free for all, anyone can participate so anyone can win. The skill of the player is unknown and therefore cannot be a factor in the formula.

Yup, and there lies the gap that's hard to bridge. One-to-one comparisons between pipsqueaks are nice and easy to make calculations about, but they don't reflect very well what goes on in a free-for-all match. (If you compete against five pipsqueaks in a race, your performance doesn't have five degrees of freedom, but only one, namely your laptime.)

(The most principled-looking approach I have seen yet to handling free-for-all matches in their own terms, without trying to reduce it to 1v1 matches, is the one in this Glickman paper. The math there, however, is a little intimidating, and I couldn't bring myself to sit down and digest it properly just yet. In the meantime, I keep throwing approximations at it to see what sticks.)

14

Stunts Chat / Re: Race strength estimation, revisited

« on: January 03, 2021, 09:02:55 PM »

Have you been here already, Duplode? https://arxiv.org/pdf/2008.06787.pdf

No, I hadn't; thanks for pointing me to it! This paper was posted on arXiv in August, while most of my research was done between April and May. I see at least three nice things in it:

• An evaluation of various metrics for evaluating the rating systems, which could be a huge boon when it comes to tuning the system parameters.
• A concise presentation of TrueSkill, which I might want to try adapting at some point in the future.
• The way the results suggest all systems are wrong, only in different ways. Besides being psychologically reassuring, that squares with my observation from May that replacing my Elo engine with a Glicko implementation didn't actually improve things.

One interesting detail I noticed is that the paper's adaptation of Elo to free-for-all, described in section III.A, involves calculating winning probabilities by summing pairwise victory probabilities against all pipsqueaks and dividing by the number of pairwise combinations. That looks like a pretty rough approximation! (If I'm reading it right, it's as if each pairwise match were equally likely to decide the winner, which might not be too far from the truth if everyone's ratings are close to each other.) At any rate, unless I'm missing something it doesn't seem that much more sophisticated or principled than whatever it is I have been doing here. It's a tricky problem, really.

15

Custom Cars with Stressed / Re: Pontiac Firebird Turbo Trans Am - K.I.T.T.

« on: January 03, 2021, 05:29:33 AM »

Fantastic! This KITT is going straight to the Southern Cross collection as soon as I get the chance   

The pursuit mode is truly on demand: you can choose to turn it off entirely (by using auto gears), and even in manual you have a lot of control over when it will be engaged. One extra thing the pursuit mode could enable is a sort of drifting challenge: take a slow dirt track, or perhaps a ZCT77-style off-road course, and race it on manual with only the first gear -- the catch being that if you ever drop below 50 mph you'll lose several seconds in order to reengage pursuit mode.