Stunts Forum

2023-04-22 update: The proposal below is being updated to better reflect our current understanding of unstable replays. To avoid invalidating the next several replies, removed passages will be struck out, while additions will be in italics. For a rundown on when instability arises and how to work around it using the replay controls, see this post (https://forum.stunts.hu/index.php?msg=89637).

---

I'd like to start a discussion about the unstable replays rule, motivated by the replay invalidations at ZCT260 (see the shoutbox posts from April 8th and 9th (https://zak.stunts.hu/index.php?page=newsarc#day-8) for what happened there, and the end note at the bottom of this post for an explanation of what unstable replays are). I will open it by putting a rule change proposal on the table for your (and dreadnaut in particular, of course) consideration.

The proposal

Change the relevant passage of the rule which makes unstable replays invalid (rules page (https://zak.stunts.hu/rules/2023), section "The System", bullet point #2) from:

"[...] and [a replay] should play correctly when loaded in Stunts, from 'Options → Load replay'."

To:

"[...] and [a replay] should show as complete upon loading in Stunts from 'Options → Load replay', rewinding to just before the finish line, and continuing from there."

2023-04-22 update: While checking how the replay ends upon loading from Options should be the primary means of validation, the current procedure of watching the lap from the beginning, with multiple cameras if necessary, should be kept as a fallback. That being so, it might be appropriate to concisely describe the validation procedure and how it is supposed to work in a separate item.

Rationale

My proposal tries to strike a compromise, keeping replay viewing and validation reasonably easy while excluding as few valid-as-driven replays as possible. To expand a bit on that, I'll consider the two goals which, as far as I understand it, motivated the 2021 rule change:

• Ensuring competition management can validate replays by following a simple procedure with reproducible steps.
• Avoiding confusing replay watchers who play laps from the competition archives.

Goal #1 is fully addressed by my proposal. In fact, as far as manager convenience goes it even improves upon the status quo, as it is no longer needed to watch the whole lap to be sure it is valid -- just loading from Options, rewinding a little bit, and continuing to the evaluation screen is enough. (This streamlined procedure is what I have always assumed managers short on time would default to on race closing, which partly explains my mix-up when replying to Friker at the shoutbox on April 2nd.) Besides that, unstable replays which are valid according to this procedure (i.e. the vast majority of them) can be watched to the end on normal speed through the well-known replay controls trick of rewinding a little around shortly after the point of divergence and resuming play, and are reproduced correctly by the repldump tool, and usually, with an adequate choice of starting frame, when played at double speed as well. (And conversely, Overdrijf's extraordinarily hard to verify replay from ZCT232, which triggered the rule change, fails the streamlined verification.)

2023-04-22 update: Considering what we have learned over the past two weeks about unstable replays -- in particular, that divergent timelines are system dependent, and that replays such as Overdijf's ZCT232 one aren't as rare as previously thought -- it makes sense, as suggested by Friker, to keep the current, play-from-the-beginning validation as a fallback method. It is fairer to drivers to accept as many unstable replays as we reasonably can, and keeping the current method as a fallback won't noticeably increase admin workload. The main caveat is that, since divergent timelines are system dependent, if a replay is only finished on a divergent timeline and not on the load-from-Options/fast-forward one there is no guarantee that the manager will be able to watch the lap being completed. That being so, it seems appropriate to keep the new wording of the rule as it was in the original proposal, recommending pipsqueaks to check if their laps show up as complete upon loading from Options -- if they don't, there is a nonzero chance of them being ruled invalid, even though the fallback validation might rescue them.

As for goal #2, there are two aspects to consider. Firstly, there being a reproducible way for managers to validate replays means every one else can check them in some way, and as pointed out above that also translates to reasonable ways of actually watching the laps. Secondly, I feel we should not penailse, in multiple ways, pipsqueaks in the here and now for the hypothetical benefit of a future visitor who might be confused by archival replays -- specially considering that the pipsqueaks have valid-as-driven laps and are at no fault, and that we can provide easy access to information that clarifies the matter to visitors (through Wiki articles, replay package readmes, and so on).

If need be, I can go into further detail on the points above, and in particular on what makes it so taxing to verify replays in the way specified by the current rule. But I have talked for long enough for an opening post, so it's time to give the floor to you. 

---

End note: what is an unstable replay?

An unstable replay is a lap, usually a RH powergear one, which plays differently in-game depending on how the replay controls are used. In the most common scenario, the player completes the lap normally without noticing anything strange, and the replay shows up as complete upon loading from the Options menu and looking at the end of the tape. However, rewinding to the beginning and playing at normal speed shows the car as crashing or veering off track.

So, I would like to repeat the question from ZakStunts's Shoutbox: If you would start a new drive and input exact same inputs as in a replay - do you think/know that result of that driving is such 1. exactly as it would play in a replay from beginning/2. exactly as last position of replay when loaded from a file?

For me the answer is a key for my stance of this topic. I would like to have a rule in accordance of what would happen if the input from a replay was driven during gameplay (and I silently hope it is the second option :) ).

Until the confirmation of what would happen I would like to relax the rule to Overdrijf's proposal - "counting any replay that can be played in any way as valid".

It is option two. But, when you are Replay handling you basically are constantly time traveling. So the game has to recalculate from the point you enter the timeline. That can give a different result because of a rounded position calculation at that point. That can produce a slight offset, that is different when replaying the replay.
That little difference can be the difference between finishing and crashing.

Quote from: Daniel3D on April 10, 2023, 05:09:11 PMIt is option two.

Are you sure? How do you know? That would mean that current rule does not match "reality" - state that would be reached if "a replay would be driven".

Quote from: Daniel3D on April 10, 2023, 05:09:11 PMBut, when you are Replay handling you basically are constantly time traveling. So the game has to recalculate from the point you enter the timeline.

I was under impression that when you are doing RH the position is recomputed always from the beginning - at least when you rewind backwards (that's the reason why it is slower to rewind than to move forward).

Quote from: Daniel3D on April 10, 2023, 05:09:11 PMThat can produce a slight offset, that is different when replaying the replay.
That little difference can be the difference between finishing and crashing.

What exactly is producing a slight offset and a slight offset related to which other event?

I am sorry about asking so much but I would really like to understand different ways of simulation/how Stunts engine works.

You create a cut at the position you stop. For normal gameplay that is not an issue because the game is precise enough at that level. In a high speed Pg lap it could be not perfect.

Quote from: Friker on April 10, 2023, 03:43:19 PMIf you would start a new drive and input exact same inputs as in a replay - do you think/know that result of that driving is such 1. exactly as it would play in a replay from beginning/2. exactly as last position of replay when loaded from a file?

I'm a bit confused about this question, doubly so as Daniel mentioned replay handling which didn't seem to be what the question was about 🤔

I think the answer is "always 1, sometimes both". Entering movements (driving!) via keyboard, mouse, joystick or replay file appears to be equivalent. The fast-forward option used when loading a replay (without the lap actually appearing on screen) seems to handle things differently: when you load a replay and see the end result*, some unstable replays show a crash screen.

(*) with some actions you load the the replay and see the starting line instead

Quote from: Duplode on April 10, 2023, 02:48:47 AMI'd like to start a discussion about the unstable replays rule, motivated by the replay invalidations at ZCT260

Thank you for starting this thread Duplode, and for the thorough prososal. There is one bit missing though, which I think is important to direct the discussion: what is the problem we are trying to solve with this rule change?
 

Quote from: dreadnaut on April 10, 2023, 06:43:56 PM

Quote from: Friker on April 10, 2023, 03:43:19 PMIf you would start a new drive and input exact same inputs as in a replay

I'm a bit confused about this question, doubly so as Daniel mentioned replay handling which didn't seem to be what the question was about 🤔

Quoteassumption is the mother of all fuckups

I assumed the replay in the example being a RH PG replay, because the original proposal is based upon an issue that is (exclusively?) related to RH PG replays.

And I may read/understand the question wrong.

Although I have already benefited from a loose interpretation, I still think that a valid replay should run completely from start to finish. But we need a well-defined way to determine that.

My solution would be a closed and clearly defined reference system on which the replays are played. Only if they run completely through there, the replay is valid. The reference system should reflect as well as possible an installation of stunts on a real computer of that time. Nobody knows on which system we will play in 20 years and therefore it is good to orientate on the original - that will always be the reference.

In my opinion DOSBox is out of the question, because it is not very close to the hardware. Better would be the very accurate emulator PCem (but only because it can hardly be realized with a real computer :) ). The perfect solution for me would be a virtual system, to which the replay is transferred via an API. It then outputs whether the replay is valid or not. Maybe even with a video. The whole thing could then also be used in a competition so you know short time after the upload for sure if the replay is valid.

I understand the point that it's difficult to spot mistakes while creating the replay. But just like, for example, sometimes a penalty-time isn't spotted until too late after a difficult passage, that's part of stunts for me, too.

Whatever we decide in the end, I think it is important that there is a clear procedure / rule with which everyone can live and which leaves no more room for maneuver, so that we don´t need this discussions about such replays anymore. As long as the system described above does not exist, I am for the current rule.

Quote from: dreadnaut on April 10, 2023, 06:43:56 PMEntering movements (driving!) via keyboard, mouse, joystick or replay file appears to be equivalent. The fast-forward option used when loading a replay (without the lap actually appearing on screen) seems to handle things differently.

How do you know that the first sentence is true? That is what I am asking - there are two options - either driving the same inputs as in a replay produces output either 1. as during replay (load replay, rewind to beginning, press play) or 2. as during "fast-forwarding" (even with one-step/0.05s forwarding - load replay, rewind to beginning, press fast-forward for one frame, repeat pressing fast-forward for one frame until finish). The point of my question is that I would personally prefer if the final ruling on valid replays was based on (what would be the result of) inputs from replay driven during a new drive (because ultimately they were driven and input during racing). That would result in: if my 1. option is true then I would agree with current ruling, if my 2. option is true then I would agree with the new proposal (load replay+rewind 1.05s).

But for that I would like to know how engine works. Not only guessing. We as community have Stunts code reversed so I would assume that there are some folks who know answer for this question for sure/can debug code to prove either option. (And I would be glad to learn this magic so if you can teach me I am ready to listen. :) )

Quote from: Frieshansen on April 10, 2023, 07:28:32 PMBut we need a well-defined way to determine that.

I would say if the folks who know how Stunts code works answer than we would have pretty deterministic way. I have pretty strong feeling that if it is a rounding error/overflow this is either 100 percent reproducible on all machines (it is some kind of forgotten register which is not re-set in assembly when you press play instead of fast-forwarding) or 100 percent different on different real-life machines (I can only assume different execution of operations in a given processor/overflowing of registers on different systems) thus there is no "ultimate true system" (even with PCem) so we would end up with one arbitrary chosen and in that case I would choose DOSBox packed in official .zip file not because of accuracy but because of accessibility - even checking if a replay is valid on my computer was PITA - handling it via some API would be ridiculous for me.

Quote from: Daniel3D on April 10, 2023, 07:28:21 PM... an issue that is (exclusively?) related to RH PG replays.

Maybe there are some high speed magic carpets which behaves differently?

Quote from: Friker on April 10, 2023, 08:35:47 PMHow do you know that the first sentence is true?

Because it contains the word appears ;D  Jokes aside, I would expect the code to have one "game loop" that connects inputs, timings, simulation, and graphics code. The fast-forward feature of the game remove timings and graphics, and joins together (replay) inputs and simulation. That's likely to be a separate loop, which can therefore behave differently. The alternative (both fast-forward and game are handled by the game loop) would require a bunch of conditions in the game code, which would make it more complicated and slower — it's possible of course, but seems inconsistent with the optimisation practices of the 80s and early 90s.

Quote from: Friker on April 10, 2023, 08:35:47 PMWe as community have Stunts code reversed so I would assume that there are some folks who know answer for this question for sure/can debug code to prove either option.

I would compare our current knowledge to: we took apart an amazing mechanical clock, and we are looking at the single gears, or small sets of gears. That is quite far from understanding how the complications and movements behave in the live clock, but it's the starting point to get there.

The single pieces of code are meant to do one things, but how they all interact together to create two different results... I don't think we understand that yet.

Quote from: Friker on April 10, 2023, 08:35:47 PMI have pretty strong feeling that if it is a rounding error/overflow this is either 100 percent reproducible on all machines (it is some kind of forgotten register which is not re-set in assembly when you press play instead of fast-forwarding) or 100 percent different on different real-life machines (I can only assume different execution of operations in a given processor/overflowing of registers on different systems) thus there is no "ultimate true system" (even with PCem) so we would end up with one arbitrary chosen and in that case I would choose DOSBox packed in official .zip file not because of accuracy but because of accessibility - even checking if a replay is valid on my computer was PITA - handling it via some API would be ridiculous for me.

My problem with DOSBox is that it plays some replays differently on different computers despite having the same version and config. That's why it's out of the question for me for a predictable system.

It seems I did succeed in starting a discussion -- great!  :D

Quote from: dreadnaut on April 10, 2023, 06:43:56 PMThank you for starting this thread Duplode, and for the thorough prososal. There is one bit missing though, which I think is important to direct the discussion: what is the problem we are trying to solve with this rule change?

In ZCT260, we have seen laps from two pipsqueaks being deleted even though they were valid as driven. (In what follows, I will use "valid" as shorthand for "valid as driven" -- more on that in a moment.) I believe we should strive to have as few valid laps deleted as we reasonably can. In particular, if there is a simple validation procedure that doesn't report invalid laps as valid, and reports valid laps as invalid far less often than the current procedure, we should switch to using it -- ergo, my proposal.

The concept of "valid as driven" that I'm using is a really simple one. It is certain that, when Friker drove his deleted lap, he saw the evaluation screen at the end of the session, and it said "Elapsed time: 1:56.95". If we were sitting by his side at that moment, we'd accept that was a valid result, and no one would care about how the replay would be reproduced, or any of the technical minutiae we've been poring over here. In brief: the evaluation screen shown to the driver is the ultimate source of validity built in the game, and we should honour it to the maximum possible extent.

Relative to the evaluation screen, the replay file is secondary, derived, accessory -- a means of transmission, useful for when you can't be in the same room as your fellow pipsqueaks. Replay files are pretty reliable overall, but not completely so: watching an unstable replay will sometimes mislead one into thinking a lap driven to the finished by a pipsqueak was left incomplete. Fortunately, we are able to easily detect most such inaccuracies, which is what I'm suggesting that should be done.

Quote from: Friker on April 10, 2023, 03:43:19 PMIf you would start a new drive and input exact same inputs as in a replay - do you think/know that result of that driving is such 1. exactly as it would play in a replay from beginning/2. exactly as last position of replay when loaded from a file?

We don't know for sure, and it's likely that will only change when the code of the game gets analysed in enough detail for us to find exactly what causes the underlying bug. repldump (https://forum.stunts.hu/index.php?msg=54243) plays unstable replays correctly (that is, to the finish line). That, however, is not enough to be sure that #2 holds: repldump only includes the core parts of the game loop needed to update the gamestate data structure, and as dreadnaut notes the bug might lie in an interaction with something outside of this core.

In any case, there is a deeper issue at play here. I don't think the hypothetical verification procedure you suggest (starting a new drive and entering the exact same inputs as recorded in the replay file) is actually the gold standard we should abide to. Since ZakStunts is an RH competition (as @Daniel3D has underlined), that procedure is not completely faithful to how the lap was actually driven, as it treats all replays as if they were driven without RH. If, in the course of an RH racing session, a timeline divergence affects a lap that the pipsqueak will eventually drive to completion, the divergence is an integral part of the gameplay, and not merely an artefact of replay watching. What matters at the end of the day is that the pipsqueak completed the lap and reached an evaluation screen displaying a lap time, thus making it valid.

Quote from: Frieshansen on April 10, 2023, 07:28:32 PMMy solution would be a closed and clearly defined reference system on which the replays are played. Only if they run completely through there, the replay is valid. The reference system should reflect as well as possible an installation of stunts on a real computer of that time. Nobody knows on which system we will play in 20 years and therefore it is good to orientate on the original - that will always be the reference.

In my opinion, this is not a direction we can afford moving in, for the reason Friker alludes to: most people rely on DOSBox to play, and so relying on an different and uncommon setup for validation would make racing less accessible. (Still, the issue of replays playing differently across DOSBox-running computers does sound worth investigating. Do you have any example replays at hand?)

Ok.
So, if I understand correctly.

Quoterepldump only includes the core parts of the game loop needed to update the gamestate data structure,

That is the same code the game uses upon loading a replay.

• This is the replay as driven.

When replaying the replay the game has to draw the environment and interactions again, it can treat edge case calculations different (appearendly)

• Replaying can change the outcome just as opponents behaviour can vary.

So. Loading a replay and rewinding enough to see the finish line should verify the replay "valid as driven"

Can replaydump be used to create an automated replay validator?
Check if the replay ends in the neighborhood of the Finish?
Or even check if checkpoint areas are passed?

Quote from: Daniel3D on April 11, 2023, 08:58:53 AMCan replaydump be used to create an automated replay validator?

Yes, in principle this can be done through repldump. Its output has all the information we might care about a lap, including the information needed to know if the lap was completed.

There are two caveats I find important to mention. Firstly, the result of such a validator is not truer than the in-game experience of the pipsqueak: as per my argument above, the evaluation screen rules supreme, and the accuracy of a validator is a matter of how well it can reproduce it. Secondly, to the best of my knowledge a repldump validator will give the same results as the streamlined verification procedure that I'm proposing. In any case, a repldump validator would certainly be an improvement over the status quo on this issue, as it would confirm more valid replays as such than the current procedure.

I thought of the replaydump based replay verifier as tool to aid moderation.

Hm.. there is a good argument about what a person see at evaluation screen should be valid. Then again - if it is a result shown only on somebody's machine and not reproducible anywhere else - should it be valid? In very extreme case - after memory manipulation - it can end badly as a verification tool.
Because we do not know what is causing inconsistencies I would propose to have validation procedure as Overdrijf proposed - if replay finishes in any way successfully on (let's say) 3 independent machines (pipsqueaks from 3 different teams) it should be valid. And apply this only on questionable replays (someone brings that up let's say max. 3 days after race).

How often this thing happens? Once in a year/two years? Agreeing on a difficult validation process seems not worthy to me.

Quote from: Frieshansen on April 10, 2023, 10:00:45 PMMy problem with DOSBox is that it plays some replays differently on different computers despite having the same version and config. That's why it's out of the question for me for a predictable system.

I've never heard about this. Do you have such replay/config?

Quote from: Duplode on April 11, 2023, 11:53:16 AMFirstly, the result of such a validator is not truer than the in-game experience of the pipsqueak: as per my argument above, the evaluation screen rules supreme, and the accuracy of a validator is a matter of how well it can reproduce it.

It is not any truer but from what I saw and it seems other experienced too it is not less useful than the in-game experience and having something independent seems like a step forward.

Quote from: Duplode on April 11, 2023, 11:53:16 AMSecondly, to the best of my knowledge a repldump validator will give the same results as the streamlined verification procedure that I'm proposing.

I would love to be sure that this is true. That's why I would like to see/understand/debug code/program. I assume that when you start a new drive and start driving you get the same result as while RHing and replaying the file with fast-forwarding. I would like to have that confirmed.

Because of my assumption - I think that when you press Play when replaying there is something different going on than when you start driving from menu.

Daniel's comment about driving with opponents triggered a memory supporting my assumption - I think there were cases when re-Playing a replay (saving a replay, loading, rewinding to start and pressing Play) with opponent ended with a premature crashing. But I/we did not bother to rewind a little bit to see if it is "for real". Maybe that is another case where simply Playing a replay is something different than driving. And - I think that opponent's behavior is deterministic (as a developer you would want it to be deterministic when you have a replay system). Which is another thing which would be great to verify.

Quote from: Friker on April 11, 2023, 01:06:00 PMHm.. there is a good argument about what a person see at evaluation screen should be valid. Then again - if it is a result shown only on somebody's machine and not reproducible anywhere else - should it be valid?

In practice, it is certain that if a replay shows up as complete with my proposed verification procedure (or with repldump), the pipsqueak saw the evaluation screen in the expected way. The odds against the opposite possibility are astronomical.

Quote from: Friker on April 11, 2023, 01:06:00 PMHow often this thing happens? Once in a year/two years? Agreeing on a difficult validation process seems not worthy to me.

My proposal is the simplest reliable validation procedure there is. It is done entirely in-game, and takes around 20 to 30 seconds per replay. (Doing it with repldump would be more complex, for involving an external tool, but could be even easier for the manager, as it can be automated.)

(Also, I'd say once per season is a lot for something that can remove people from races, and which ultimately is a fairness issue.)

I don't want to make decisions only towards my preferences but what is best for the community.

In one hand I'm ussualy affected by this rule and I found hours of work in a replay in the trash can. In the other hand there must be a limit to admit a replay and don't give extra work for the manager. There should be a solution between both.

Quote from: Duplode on April 11, 2023, 05:37:04 PMMy proposal is the simplest reliable validation procedure there is. It is done entirely in-game, and takes around 20 to 30 seconds per replay.

And I agree and fully support this if - if we can prove that it is the same thing as driving it from the menu or at least it is the most coherent thing in case that everything (a new drive, re-Playing a replay, fast-forwarding a replay, RH handling) behaves differently. But first I would like to know for sure/I would like to check the code.

Repldump still can be a valuable tool - similar to RPLInfo - verification before posting. In fact Repldump validation can be as a section in RPLInfo output (another feature request maybe? ;) ). :)

About the rules
I really hadn't noticed or didn't remember that, for unstable replays, the view at the last frame when you just open it is normally correct. I kind of "remembered" that even when you just opened it, it looked crashed. Now, if it really is the case that most of the time this works well, then I definitely agree that we should try this as a new rule, because it's no doubt a very simple one to use. Because I'm not very skilled at PG and my replays aren't the fastest, I don't come across with unstable replays of my own very often, but it has happened to me.

About "why" there are unstable replays. My thoughts
I have gone through the messages in this thread rather quickly, so please excuse me if there's something crucial I haven't seen. It called my attention that DOSBox may sometimes play the same replay differently with the same version?  I'd like to see that because that is something I surely cannot explain right now.

On the other hand, I can give you an educated guess on what I think causes unstable replays, from a programmer's perspective, as long as DOSBox (same version on same computer) is deterministic. My view is that this is a problem of quantisation. We see this often in digital music, in MIDI, in particular. Stunts runs a loop and every iteration of that loop does a few things. When everything you had to do is complete, it waits until the frame time is completed (1/20th of a second, minus the already elapsed time during the current iteration); then, it proceeds to the next iteration.

The problem here is that there are two tasks that occur during the first part of the loop and that are crucial. One is the moment when the input configuration is gathered and the other is the moment when this information is stored. When the input comes from a replay, there should be consistency because the same input configuration is assumed to rule during the whole frame, but when you're playing live, such as when you're recording a replay, they keys you're pressing will be recorded at some point, which is not at the beginning, nor at the end of the iteration. When this occurs depends on how fast the computer is because the length of the active portion of the iteration varies. But what if you just pressed the braking key a moment before it was recorded, or just before the end of the iteration?

Every time you convert a continuous flow of inputs into a discrete sequence, you quantise, and you're adding an error. This can't be prevented. What the game should do is, instead of taking the live input for the determination of physics, it should take the quantised results. But it's not doing it, because most of the time, the result is pretty similar.

Now this wouldn't explain why the replay looks good when you just open it. Here I have to guess that the "super-fast-forwarding" routine that places the pointer at the end of the replay when you just open it has a loop that's not quite the same as the normally reproducing one (that's also the one that's used when you fast forward with the controls). Maybe frames are quantised at the beginning in one and at the end in the other?  Anyway, although it's not the same situation as with live recording, I'm pretty sure the problem must have to do with quantisation.

Repldump verification might or might not work, depending on whether the loop used by the repldump routine is the one obtained from normal reproduction or from the fast reproduction that's done at the beginning of loading a replay. We have to try and see. It'd be very interesting to know this.

In my ideal world, any replay you can spend hours on without ever realizing it might be crooked would be seen as valid, as such my first suggestion for a rule about this would be "any replay that can be made to play in any way using Stunts 1.1 or a fully compatible program is counted as valid".

Other people have raised valid points. It would be nice if the race administrator wouldn't have to regularly spend an hour trying to get a replay to work. I've also learned from Argammon that apparently there is under some circumstances a way to use replay handling to intentionally reset to the exact moment of certain select high speed powergear crashes and continue from the frame of the crash. I didn't know that. Sure, that one borders on cheating, even if it is very situational and probably requires quite some driving skill to use well. I am therefore open to any compromise that reduces administrator time spent, reduces the possibility of anything that may be considered cheating* or helps in any other way. But my basic stance on the matter is that this is RH Freestyle, we drive on the edge, and we regularly sort of break the game in various ways to get the fastest times. Breaking a replay while doing that is just the risk we take, and I'd be happy if as many questionable but ultimately fairly raced replays as possible were able to be counted as valid.



*(On the other hand, the Freestyle RH format was born because cheating through replay handling was undetectable. There is something to be said for allowing as many techniques as possible in the format, even if they look a lot like driving straight through a crash...)

Quote from: Duplode on April 11, 2023, 06:21:08 AMIn particular, if there is a simple validation procedure that doesn't report invalid laps as valid, and reports valid laps as invalid far less often than the current procedure, we should switch to using it -- ergo, my proposal.

Props for defining unstable replays as "valid", and the arguing that we should not invalidate "valid" replays ;D  Ready for politics!

But although well explained, this is not The Issue. The real problem is human: we don't like that pipsqueaks work hard on a replay, but it ends up in the bin. This is what this rule change is about.


Now, let's define unstable replays as replays that load in a crashed state, or load in an OK state but play to a crash. Both are a thing, and I don't have statistics on whether a kind is easier to generate than the other.

My position is that invalidating unstable replays affects only a few high level pipsqueaks, and rarely, but accepting unstable replays affects everyone else, today and tomorrow:

- it is difficult to rewind / replay an unstable replay in a way that reaches the end
- the above is most problematic with "broken at loading" replays, where we have to argue for validity, and is a cause of arguments and unhappiness (see past races)
- they cannot be replayed without knowing what to do, reducing their archival value, and...
- they are frustrating for beginners (powergear AND extra bugs!)
- you cannot make videos out of them
- (any automated tool we might create in the future will have to deal with them)

Pro pipsqueaks are already used to failure and repetition, and I thought they could deal with the extra difficulty. You can drive a high level powergear lap? Sorry, there's an extra validation for you. It's easy and you can do it on your computer. I have been that driver: powergear is the only trick that I can perform at 'gold' level.


Looking at the problem again (invalidating replays is cruel), I would say — yes, it's true! Sorry, the choice is between a cost for the individual, or a cost for the community.

From this point of view, I am not interested in high tech solutions to validation. Spending time reviewing replays is not really a problem — I get to see a lot of cool stuff! What I'm interested in is "Can we reduce the cost for the community?" or "Is the community happy to accept the cost?"

Cas, I am sorry, but I completely disagree with your points. It would be a really big oversight if a variable (which keys are pressed down) which is used at least in two places (computing next position of a car and saving a replay string) could be modified in the middle of the process. If the variable is modified outside this critical section all is fine (because nothing is done at that portion of time). Furthermore your points do not explain discrepancy between fast-forwarding and playing a replay. But.. all of that text and Frieshansen's magic DOSBox behavior got me thinking..

And I did some testing. First of all, there is an sub-menu "Options" -> "Set graphic level" which contains some magic text and options I do not understand. I tried to play with them in the past - no change. But after reading last comments.. I did try to lower the DOSBox's CPU speed - to cycles=fixed 2500 (Stunts visibly stutters). Lo and behold - playing a replay with Play button (or Play Fast button) produces the same result as fast-forwarding!!! !!! !!! I was pleasantly surprised. Then I tried pretty high cycle speed (150000) so my machine does not catch up - same result as during normal speed - de-sync and crash. Then I tried to lower cycles gradually and the magic happens. And somewhere between 4000 and 5000 cycles there is a magic area where a replay plays "correctly" (that means does not crash) or fails depending on graphical settings. Higher graphical settings means a simulation is not catching up and is skipping some drawing routines - maybe. That draws me to a conclusion that when there is a pause between frames the Stunts' drawing routine somehow messes with "world's state (car rotation/car position/random number generator function/whatever)". Maybe it is enough that the drawing routine finishes once, maybe twice, maybe it corrupts the state after running 100 times (not to get into politics but also changing a government is considered a good thing).

These observations somehow fits into my imagination of what is/should be "a correct/valid replay". I am repeating myself - I think that only "weird/incorrect" behavior is produced by playing a replay with "Play" button (now I can add) with high-enough cycle speed, and that 1. driving a new drive 2. replaying a replay with fast-forward 3. playing a replay with Play button with low-enough cycle speed (4. repldump) produces the same result (and should be accepted as valid).

Btw - while testing I always rewinded/rewound my replay at around 1:33:10 - so I assume that there is no accumulative error from the beggining but rather the section from 1:38:00 to 1:46:10 produces something magical. Also - it is something very consistent/deterministic so I would assume that the drawing routing corruption is also pretty deterministic and we could spot it in the code/debugger.

Quote from: dreadnaut on April 11, 2023, 11:51:48 PMNow, let's define unstable replays as replays that load in a crashed state, or load in an OK state but play to a crash. Both are a thing

Ugh, really there exist replays which load crashed but replays ok? And you can continue driving before the finish line and they produce the evaluation screen? I want to see some to analyze them with other settings. Still, it seems my last comment holds true - the problem is with the Play button. :)

Quote from: Friker on April 12, 2023, 12:02:08 AMI did try to lower the DOSBox's CPU speed - to cycles=fixed 2500 (Stunts visibly stutters). Lo and behold - playing a replay with Play button (or Play Fast button) produces the same result as fast-forwarding!!! !!! !!! I was pleasantly surprised. Then I tried pretty high cycle speed (150000) so my machine does not catch up - same result as during normal speed - de-sync and crash. Then I tried to lower cycles gradually and the magic happens. And somewhere between 4000 and 5000 cycles there is a magic area where a replay plays "correctly" (that means does not crash) or fails depending on graphical settings.

Thanks for doing these experiments, Friker! I have tried it with my unstable winning replay from ZCT099, and it goes exactly as you describe: it plays from the beginning to the finish with 2271 cycles, and veers off-track at 20000 cycles. I don't know how y'all see it, but I feel this being a DOSBox induced (or maybe even hardware induced, who knows) issue is a strong argument for erring on the side of leniency towards unstable replays.

---

@dreadnaut: There are a few things I'd like to reply to. What follows comes across as rather confrontational, and I'm sorry about that. It does seem that we're entering this discussion from very different positions, and that I haven't quite managed to make mine clear yet. That being so, I'll try to state my priorities more directly, rather like you have done. If where I'm coming from becomes clearer, that will already have been some progress.

Quote from: dreadnaut on April 11, 2023, 11:51:48 PMBut although well explained, this is not The Issue. The real problem is human: we don't like that pipsqueaks work hard on a replay, but it ends up in the bin. This is what this rule change is about.

I agree the issue is human, and wasted effort from pipsqueaks is a very important part of it. It is not the only one, though. If you allow me an absurd example, no one would care about the wasted effort if this were about someone spending fifty hours crafting a replay byte by byte on an hex editor. Things are different, though, if we are talking about a pipsqueak who drove their lap and had the game confirm it as complete. That being so, I see this as a matter of fairness, with the root issue being the one I have described earlier. The other human (or social) issues, chiefly among them the waste of effort and the joy-killing effects of result deletion, are very important consequences of that.

Quote from: dreadnaut on April 11, 2023, 11:51:48 PMMy position is that invalidating unstable replays affects only a few high level pipsqueaks, and rarely, but accepting unstable replays affects everyone else, today and tomorrow: [...]

Stated in this way, this looks like a straightforward utility calculus. However, I don't find it so simple. The main point of a Stunts competition is offering pipsqueaks the conditions for fair and enjoyable racing. Everything else follows from and depends on that. To my eyes, your stance amounts to sacrificing the gameplay experience and enjoyment of pipsqueaks for the sake of a number of meta concerns that are either hypothetical or secondary (we can ask Alan for confirmation, but I suppose we won't be bothered by having to capture a video in two separate parts once in a blue moon), and I don't see this as a good tradeoff. Things would be different if all unstable replays were like that one lap from Overdrijf, and every powergear race became a verification nightmare, but that's clearly not the case.

I'd like to comment specifically on the archival value point, given that, being the Competition Archive maintainer, I have some skin in the game. In my opinion, it is not an issue at all. Unstable replays have been known to exist for very long, and used to be broadly regarded as unproblematic by competitions -- just another peculiarity of RH powergear racing. As a consequence, the archives already feature unstable replays, and their value is no lower because of that. In fact, it is quite the opposite: the archives would be poorer without them. Stunts is a deeply weird game, and unstable replays are nothing but a tiny part of that weirdness. There's no point in outlawing bits of the gameplay, quirks and all, just to fit some ideal of clarity.

Quote from: dreadnaut on April 11, 2023, 11:51:48 PMYou can drive a high level powergear lap? Sorry, there's an extra validation for you. It's easy and you can do it on your computer.

It's not quite the walk in the park you make it sound like. Since I don't want to bog down this reply in discussion of gameplay minutiae, I'll just point any curious readers to the shoutbox archive from April 8th and 9th (https://zak.stunts.hu/index.php?page=newsarc#day-8), where Friker and me look at the ZCT260 in-game situation in some detail.

Quote from: dreadnaut on April 11, 2023, 11:51:48 PMLooking at the problem again (invalidating replays is cruel), I would say — yes, it's true! Sorry, the choice is between a cost for the individual, or a cost for the community.

No, this is not a choice between individual and community. We pipsqueaks are part of the community as well -- even the "pro" ones. When pipsqueaks are asked to jump through hoops or to give up some of their enjoyment in the name of some greater good, there is a cost for the community too, just in a different way.

@dreadnaut and @Duplode.
Both of you speak on behalf of the community. There are a lot of factors involved and a lot of uncertainties.

Both opinions are valid.

However. Unstable replays are appearendly circumstance dependent. What I mean by that is. There is no way of knowing if (PG) replays that did verify are stable. They were just fine under the conditions set by Dreadnaut.
If changing the amount of cycles is of influence. Then I feel there is reason for investigating.

Ultimately I think that it is up to the community to state what they are more comfortable with.

For me a PG lap is magical, stable or not. Rejecting a replay based on stability is doesn't matter to me, the second or third best is still way up there. It only affects PG drivers themselves.
With that I think that Dreadnaut community argument doesn't apply to me.

I do feel the disappointment of other players and that influences me more....

@Duplode confrontation happens, it's part of discussing things one cares about. How we react and resolve the confrontation is the important bit 👍

Things are much more clear I think. It seems that I have discounted the effort (or likelihood?) of checking one own replays. For me it became rapidly a habit: if I go through a tough PG section, I save-load-play to check that I came out of it "clean". In a team, it happens naturally if you share your replays to discuss them: the first team mate who looks at your replay will spot the issue. Alan did not [have time to] share his last attempts in ZCT260, and we didn't get a chance.

The disappointment of discovering my replay is wobbly, at most a few hours after driving, is definitely lower than the disappointment of those who discover after the race is over. I thought the simple test would allow everyone to check early, but apparently it did not pick up. I feel a bit like we are relaxing a rule because pipsqueaks do not follow it, and are upset that it still applies. (Friker excluded) — that's the feeling, but I know the problem is elsewhere. Wass the rule unclear? Was the test not clear? How can we advertise things better?  — is it possible to improve the situation without removing the rule? Or does the rule have to go or be relaxed for this to ever work?

Quote from: Daniel3D on April 12, 2023, 09:09:29 AMIf changing the amount of cycles is of influence. Then I feel there is reason for investigating.

Maybe we need to ask Stan or Marco to play samples of unstable replays (play to crash, load to crash) on an old 386-486 PC, and see how they behave on original hardware 🤔
 

Quote from: Duplode on April 11, 2023, 06:21:08 AM(Still, the issue of replays playing differently across DOSBox-running computers does sound worth investigating. Do you have any example replays at hand?)

Attached are two of my replays with this problem from ZCT251. I tried to collect all available data - thanks to @dreadnaut and @alanrotoi for the support. All information is from 06/2022.

1-17-50.rpl      
dreadnaut:
<= 6000 cycles: completes
>= 7000 cycles: crashes in the cork r/l just before the finish line
alanrotoi:
completes
frieshansen:
completes

1-17-45.rpl      
dreadnaut:
<= 6000 cycles: completes
>= 7000 cycles: crashes into the tunnel just after the cork l/r
alanrotoi:
completes
frieshansen:   
completes

Systems

dreadnaut:
DOSBox staging 0.76.0, DOSBox staging 0.78.1 -> all on Linux (not 100% sure about the OS anymore)
with the attached config dosbox-0.76-stunts.conf

alanrotoi:
DOSBox 0.74-3 with cpu speed 20000 cycles

frieshansen:   
DOSBox 0.74-3, DOSBox staging 0.76.0, DOSBox staging 0.78.0 -> all on Windows 10
DOSBox staging 0.76.0 -> on Linux (in a VM on Windows)
all with config file from dreadnaut


I tried a lot of things back then to somehow recreate the problem on my system, but failed with everything. At this point I lost confidence in a reliable replay by DOSBox and thought that the final decision about a valid replay should be decided with the help of the most accurate possible instance of the game itself. I have no problem with DOSbox per se and it is certainly the easiest way for us to drive, but for validation in extreme cases, I think it is poorly suited.
But I'm probably going a bit overboard here. If even the gurus of stunts like Duplode hardly know this problem, it is probably very rare. I also can't say for sure if it's really DOSBox - it's just a guess for which I have no proof.

Wow, I have missed a lot of posts while I was away. Without further ado, here is some more research:

I found that the problematic frame in Friker's replay is 1.46.00, in which the car completes the sharp turn. All pictures below are generated from the same replay:

Picture 1: Press play button from 1.46.00 or before
Picture 2: Load the replay from options / Press the play button or the 0.05 forward button from 1.46.05 or later
Picture 3: Press the 0.05 forward button from 1.46.00 or before
State 1 - Crash.jpgState 2 - No Crash.jpgState 3 No Crash.jpg   

So there are actually 3 different states  ::)

3 states, of which 2 don't crash.

Presumably only one state eventually finishes, so what does the Friker of the third timeline do? Park at Joe's for a cup of coffee?

I need to write a Stunts based alternative universes story.

This is the end of the third Friker.

End Friker 3.jpg

Quote from: Overdrijf on April 12, 2023, 09:07:08 PMI need to write a Stunts based alternative universes story.

A time travel story?
Why skid always wins?
In the story style of wreck it Ralph  8)

Quote from: Argammon on April 12, 2023, 09:20:49 PMThis is the end of the third Friker.

End Friker 3.jpg

A day at the beach, not bad.

But to save this topic from my own derailing attempt: great detective work! So it looks like the offending frame is not itself a near crash, rather just a powergear and steering frame? So it looks like Friker isn't doing anything we might want to ban or even anything we can point at to say "we should all not do that, and our replays will be fine".

Hello with my daily update.. So.. I went down the line of testing with PCem as it is marketed as the most faithful simulator of late 80s/early 90s PC systems. I tried Windows 95 with Pentium 133, 100, 75, MS-DOS 6.22 with 486 50MHz, 25MHz, 16MHz. For starters - I REMEMBERED CORRECTLY THAT GAMEPLAY IS MUCH SMOOTHER THAN ON DOSBOX.. (sorry for shouting I am just excited about that. :) ) Even on 16 MHz it felt that inputs are simply crispier. :) There is Win95 with internet support so if I could set up that connection maybe I will switch from DOSBox to PCem. :)

Real deal - my replay crashed on all systems - even on 16MHz. So, what to say.. I spent 4 hours to install Win95, MS-DOS, test my replay and find out that MS-DOS at 16MHz is still quicker/doing something differently than DOSBox with 4000 cycles. :)

I still would prefer that we would agree on either 1. only fast-forwarding validation 2. both "Play button" validation and fast-forwarding validation is ok as far as there are independent confirmations on questionable/unstable replays. Option 3. only "Play button" validation is still somehow distant for my understanding of how game works (that means - I am still assuming that difference between Play and FF happens because of fast computation of something - most probably something during drawing routine).

Btw - I love that story about myself riding alongside the beach (hopefully not straight into the sea) in a fast car. :D

Btw2 - For some reason - on MS-DOS - Stunts produced invalid replay - marking 20th byte from the end of the replay as 01 and not 00. Now the bigger surprise for me - Stunts showed correct time in the evaluation screen. :o Maybe it is not surprising.. It shows that Stunts is checking car position not replay length. :)

@ all testing done - thank you guys. :) Really, thank you. I think it only shows that - if we are using Play button validation - questionable replays should be verified on different systems/choose one independent system on which we will verify and which is available for verification before the end of a race to all of us.
If we are using fast-forwarding everything replays the same on all systems and there is no difference between systems (at least PCem with various hardware simulation and DOSBox).

@dreadnaut On your closing question, you can probably guess my answer: the rule should go, because it adds busywork, inconvenience and frustration for essentially no benefit. But I've already said my piece, and there's no need to rehash it.

What I'd like to focus on now is how @Frieshansen 's experiments provide a new and very strong argument in support of changing the rule. Given that replay instability depends on the system/DOSBox configuration, we are bound to get situation in which a replay plays normally on the pipsqueak's machine but becomes unstable on the manager's. That being so, the obvious fair approach to this possibility is accepting unstable replays, and using a verification method which rules them as valid (or at least a large majority of them, if perfect accuracy turns out to be impracticable). I still believe the procedure described in my initial proposal is, all things considered, the best option we have at the moment, though that might change as our understanding of instability advances.

@Argammon , Frieshansen and @Friker : good work on those experiments, they are all very interesting and highly relevant. Also, thanks for the replays, Frieshansen -- I've began assembling a collection, and they will be in it  :) By the way, both of them play normally here (DOSBox Staging 0.80.1, Arch Linux), at both 20000 and (following Friker's lead) 2241 cycles.

I don't know if it helps, but I can run tests on old hardware, a Pentium 266, with Windows 98.

@Erik Barros It sure can help  :) From what we have seen so far, my guess is that instability will still exist on a system like that (actual DOS on a machine that's not too slow, in relative terms), though it will not necessarily show up in the same replays -- I do wonder about those Frieshansen laps!

tomorrow I should be able to download the replays from @Frieshansen and check it, as well as the replay from @alanrotoi from the last race, with the dosbox on my machine the car broke down

Quote from: Friker on April 12, 2023, 11:16:30 PMReal deal - my replay crashed on all systems - even on 16MHz. So, what to say.. I spent 4 hours to install Win95, MS-DOS, test my replay and find out that MS-DOS at 16MHz is still quicker/doing something differently than DOSBox with 4000 cycles. :)

Wait, 4000 cycles, or 40,000 cycles?

In that first case I'm not surprised the other option is smoother, but I am surprised about how well you drive on Dosbox.

Quote from: Overdrijf on April 13, 2023, 05:17:46 AMWait, 4000 cycles, or 40,000 cycles?

4000 cycles. For me, 4000 cycles is enough to Stunts go slower and skipping some frames (hopefully only from drawing). That is the difference between not crashing (below 4000) and crashing (above 4000) of unstable replays for me.

Quote from: Overdrijf on April 13, 2023, 05:17:46 AMIn that first case I'm not surprised the other option is smoother, but I am surprised about how well you drive on Dosbox.

What do you mean?

Btw - maybe it is not smoother but something definitely feels different. As I have no reflexes so I cannot tell if it is input lag or anti-aliasing/graphical interface or nothing and I am just imaging things. :)

Quote from: Friker on April 13, 2023, 08:37:33 AM

Quote from: Overdrijf on April 13, 2023, 05:17:46 AMWait, 4000 cycles, or 40,000 cycles?

4000 cycles. For me, 4000 cycles is enough to Stunts go slower and skipping some frames (hopefully only from drawing). That is the difference between not crashing (below 4000) and crashing (above 4000) of unstable replays for me.

Then I am curious. If you save a replay at 4000 cycles. Will it set 10 or 20 FPS.
And if you continue your unstable replay after the anomaly and save it again at 4000 cycles.
Will it change the replay from 20 to 10 FPS?
And fix it for normal cycles??

Ok, I did another experiment, and it gets even worse. I managed to create a replay in which the car finishes with two different finishing times! The evaluation screen occurs in both cases.  :o

Here is how you find them:

1) Pressing "play" from the beginning causes a crash on the bridge.
2) Using the Duplode method (Load from Options, go to the end of the replay) shows the car finishes in 1.57.95 plus 9 seconds penalty time.
3) Go to frame 1:46:00, press the 0.05 forward button until frame 1:48:00, and press play. The car finishes in 2:20:00 without penalty time.

Comments:
- The methods described in 1-3 are sufficient but not necessary to produce the different endings. There are other ways to produce them, in particular Ending 3).
- Ending 3), which cannot be confirmed with Duplode's, method is slower than ending 2, which can be confirmed with his method. However, I can produce a replay in which Ending 3) is faster than Ending 2).
- Duplode's method may still be better than the status quo. In particular because, according to @Frieshansen's research, it can happen that a replay crashes on the manager's but not the pipsqueak's computer when using the play button. I find that unacceptable. I am hoping we are able to find a better method though.  :) 

FRITEST.RPL

Quote from: Friker on April 13, 2023, 08:37:33 AMWhat do you mean?

I thought you said you wanted to switch to PCem because it runs the game better than Dosbox, but you were merely saying that a certain low setting on PCem runs better than a low setting on Dosbox.

Quote from: Argammon on April 13, 2023, 10:05:52 AMGo to frame 1:46:00, press the 0.05 forward button until frame 1:48:00, and press play. The car finishes in 2:20:00 without penalty time.

This is a crash for me at 1:47:95.  :-\

The method that is most stable in it's outcome regardless of the setup used is in my opinion most fair. PG is unstable. That makes it a challenge in competition. We will have to accept that the replays can also be unstable.. (maybe they all are unstable but just passed this test)

Quote from: Argammon on April 13, 2023, 11:08:00 AM

Quote from: Friker on April 13, 2023, 10:48:23 AM

Quote from: Argammon on April 13, 2023, 10:05:52 AMGo to frame 1:46:00, press the 0.05 forward button until frame 1:48:00, and press play. The car finishes in 2:20:00 without penalty time.

This is a crash for me at 1:47:95.  :-\

Can you provide a GIF?

PG can differ for everyone....
It's the butterfly effect..

I made a GIF for @Friker. Here it is: (Ending 3)

EdIt: It is quite slow so you have to wait a bit before it starts. Sorry!

Ending 3.gif

Can you do it for the other ending as well??

Ah, that's what you meant by making a replay with three different endings. You replay handled to get the alternatie ending, played to where the regular run had finished to preserve that ending, and then drove the car to the finish.

By the way, I don't think I've ever seen such a good impression of the Blues Brothers claiming a parking spot as Friker's accidental driving there.

Ok, here is more research. Please note that I am not doing this to prove Duplode's method is bad, but to gain a better understanding of how the engine works.  :)

I have created a replay that shows a crash both with the play button and Duplode's method. However, I actually finished the lap, please see the gif below. The replay is also attached.
If I can do this on purpose, it can also happen by accident when using replay handling.

load_005.gif

FRITEST2.RPL

Edit: Another relevant question is whether there are replays that show as a crash with Duplode's method but play correctly when using the play button. I did not test it but think that is the case.

Quote from: Argammon on April 13, 2023, 04:24:10 PMIf I can do this on purpose, it can also happen by accident when using replay handling

What knowledge do you have when you can do this on purpose? :D This actually reminded me of Renato Biker.. He produced magic carpets seemingly out of nowhere..

I probably messed up something - for me fast-forwarding is producing the same result as playing.. :-/ I somehow assumed that (fast-)rewind and fast-forward are the same things.. Can someone confirm that they compute car state in different manner (something like - rewind recomputes car state from the beginning without graphics included, fast-forward computes "next frame" - same as with Play button)?

Despite recent findings I still think that only suitable validation methods (accessible and quick) are 1. load a replay, rewind to the beginning, press play 2. load a replay, rewind 1.05s, see if car crosses the finish line next frame (probably continue ride with input from replay and see if the evaluation screen shows up is more precise?) 3. repldump. If you produce a replay in which you have to rewind some, play some, then continue to finish then bear the consequence that other cannot validate it. :)

Also - to stay on-topic - is there more testing to do regarding choosing validation process for ZakStunts? I feel that we (especially Argammon) showed that we can screw the game engine in multiple ways and there is no true interpretation of what is the intended way (for me personally - I would disregard any graphics and use repldump/rewind functionality only but I understand that graphical presentation is also a part of the game). So - can we finalize some options what validation process to choose or do you think that we can come up with more insights on this topic with more testing (what specifically do we want to achieve?)?

Quote from: Friker on April 13, 2023, 05:26:43 PMI probably messed up something - for me fast-forwarding is producing the same result as playing.. :-/

Playing with this bug is very dependable on the setup used. So, don't be dismayed if you can't reproduce the anomaly...

Quote from: Daniel3D on April 13, 2023, 05:31:02 PM

Quote from: Friker on April 13, 2023, 05:26:43 PMI probably messed up something - for me fast-forwarding is producing the same result as playing.. :-/

Playing with this bug is very dependable on the setup used. So, don't be dismayed if you can't reproduce the anomaly...

Maybe or maybe not. @Friker: If you post a gif of what you did, I can tell if you made a "mistake". I also agree that 1) together with 2) are currently our best bet. That way, we would be able to verify most but not all replays.

At the risk of stating the obvious, let me elaborate a bit on what I found (using my system). After frame 1.46.00, the car in Friker's replay is following 3 different paths simultaneously. By using the replay controls, you can access and drive each path separately. For example:

- If you finish path 1 (the one that currently crashes into the bridge), the car reaches the finishing line when you click the replay button, but Duplode's method fails.
- If you finish path 2, the replay button crashes the car, but Duplode's method works.
- If you finish path 3, both the replay button and Duplode's method are not suitable to verify the replay. 
- To fix the replay, you have to start driving a bit before 1:46:00 and the anomaly will likely vanish.


I hope that made sense!  :)

Quote from: Argammon on April 13, 2023, 06:57:36 PM

Quote from: Daniel3D on April 13, 2023, 05:31:02 PM

Quote from: Friker on April 13, 2023, 05:26:43 PMI probably messed up something - for me fast-forwarding is producing the same result as playing.. :-/

Playing with this bug is very dependable on the setup used. So, don't be dismayed if you can't reproduce the anomaly...

Maybe or maybe not. @Friker: If you post a gif of what you did, I can tell if you made a "mistake".

Sorry, I do not know how to easily create a gif so you have captured video from dosbox as an attachement.
I am also attaching some other of my crashed replays for others who might be interested.

Quote from: Argammon on April 13, 2023, 06:57:36 PM- To fix the replay, you have to start driving a bit before 1:46:00 and the anomaly will likely vanish.

The tricky thing in this situation is that at this point Friker is doing some very precise sliding to end up on the road rather than in the water. Starting a bit before 1:46 he'd be likely to replay handle himself into almost the same set of key presses and almost the same position. So... could that make the same anomaly appear?

Two quick comments on some of the things @Argammon has noticed:

• Even in the presence of "butterfly effect" finishes with the timelines giving different lap times, we can carry on verifying lap times like we've always done, through the automated replay length and completion checks in the site. That makes sense, in particular, because the twenty 00 bytes at the end of a complete replay file provide evidence that the lap time reported by the site was the one seen by the pipsqueak before saving the replay.
• While it's indeed possible to intentionally finish unstable replays in such a way that they will appear incomplete when loaded from the Options menu, my working hypothesis is that in practice such a thing will happen far less often than unstable replays which show as complete. I am currently reviewing examples of unstable replays so that, among other things, I can present some extra evidence on that.

If the game loads the end state of a replay consistent when loading from the options menu, regardless of who checks it. The it is good practice to check before submitting if that happens. Those kind of situations are rare and because they are easy to verify for the player.

I'd like to share a few more interesting results on unstable replays, from investigations that were being prepared over the last few days.

Firstly, @Erik Barros (thanks a lot!) has kindly ran tests on some of the replays using his bona fide Pentium 266, running Windows 98:

• Alan's deleted ZCT260 lap: loads properly from Options, crashes under the u/d cork when played from the beginning, as described here.
• Frieshansen's 1:17.50 at ZCT251 from upthread: loads properly from Options, crashes when played from the beginning.
• Frieshansen's 1:17.45 from upthread: loads properly from Options, veers off-track by the final u/d cork when played from the beginning, which is a different divergence from what had been described here! Much like in Argammon's experiments, there's also a third possible timeline, reached by rewinding to a specific spot, in which the car flies high in the sky instead of completing the lap.

These results confirm that instability is not just an emulator issue, but also affects the game on period-appropriate hardware. Additionally, the novel timelines with Frieshansen's 1:17.45 further underline that it's anybody's guess whether and how instability will be observed on any specific system.

Secondly, I have reviewed GTO, Vette and Indy winning replays from the Competition Archive in search of further examples of unstable replays, and found 13 of them. The replays themselves and a spreadsheet listing them are attached below. With the caveat that this is what I've seen on my system, and your mileage may vary, here are some notes:

• The breadth of examples makes it clear unstable replays are not a new phenomenon, nor it is restricted to specific pipsqueaks. Whenever there are lots of powerslides, there is a nonzero chance of instability -- from a driver's perspective, that's all there is to it.
• While thirteen replays might not look like a lot (we have north of 200 plausible candidates in the Archive), keep in mind these only include winning replays, and there is no reason for instability not to happen further down the scoreboard. It's not surprising that in the last 36 months (ZCT225 to ZCT260) we've had three instability incidents, out of fourteen races with a PG car on the podium and seven full PG podiums.
• Almost all of those thirteen replays load properly from the Options menu, and diverge when played from the beginning. The sole, glorious exception is Gutix's ACT13 replay, which does the opposite. While this observation supports my hypothesis that most unstable replays in competition will pass my proposed validation, I find @Friker 's option #2 from this post (https://forum.stunts.hu/index.php?msg=89486) (keeping playing-from-the-beginning verification as a fail-safe for replays which don't pass it) a very sensible one.
• The DOSBox cycles limit below which laps have the same outcome when played from the beginning as they do upon loading from Options changes per replay. While a large share of the thirteen replays here do not diverge at ~2200 cycles, that is not low enough for e.g. Alain's famous Eger replay, whose threshold here was ~1500.

I plan to do further analysis on these replays, as well as on the other ones shared on this thread, including running repldump on them all -- the working hypothesis being that repldump output always matches the load-from-Options outcome.

This investigation is going quite deep, nice! ;D

I suspected replays would crash on original hardware, and completing successfully on low-cycles DOSBox might be the simulation artefact here.

I have one more question: can we generate an unstable replay without using the "forward" button of the replay controls?

My hypothesis is that the bug (or butterfly) hides in the non-graphic play loop. If that's the case, rewinding a replay would not cause any issues, but advancing during a powergear stretch could make the car behave differently.

There has been a lot of discussion on this topic, which has been quite interesting to read.  Personally, I don't have an issue with the current rule.  I always ensure that my replays play back correctly when loaded from the options menu before posting them, regardless if I used power gear or not.  We've experienced unstable replays at Race For Kicks as well and have a similar rule for replay playability:

All replays must be playable continuously from start to finish when loaded in Stunts by selecting "Load replay" from the options menu. Playback at either standard or double speed is acceptable provided switching between the two speeds is not required for the replay to play back successfully.

Specifically, Duplode's replay for Joe's (http://www.raceforkicks.com/index.php?page=track&track=joes) track in the permanent competition will play back successfully at double speed, but not at standard speed.  We also added an icon to easily identify replays like this.

I can also perform testing on legacy hardware, if necessary.  I have a 486 and a Pentium III system that run DOS natively, so let me know if running tests on either of these would be of any help. :)

@KyLiE As it stands, the main issue with the current rule is that sometimes it will be impossible for the driver to ensure the replay plays back correctly, as the bug won't happen on their computer, but will on someone else's. This has already happened in practice with Frieshansen at ZCT251 (https://forum.stunts.hu/index.php?msg=89480). (I still stand by my earlier arguments, but this is much starker.)

I like the idea of an indicator in the site for replays that are known to be unstable. If the rule changes and such a feature gets implemented, I volunteer for reviewing laps from the past.

And yeah, more tests on legacy hardware are always welcome!  :)  The #1 test to do, I think, would be with Frieshansen's ZCT251 replays linked to above, as we're seeing a lot of variation from system to system with them. But the deleted ZCT260 replays (https://forum.stunts.hu/index.php?msg=89414), the Competition Archive ones I attached a couple posts above, or anything else you feel like trying are fair game, too.

---

@dreadnaut My current understanding is that the non-buggy timeline is the fast-forward/load-from-Options one, as that is the one which remains consistent across tests, while the watching-with-graphics timeline can change across systems and be manipulated through DOSBox cycles and graphics settings. As for legacy hardware versus DOSBox on low cycles, I think what happens is that you need to run the game on very slow hardware to eliminate the divergences from normal playback, and even legacy machines are often too fast for that. For instance, the configuration menus of DOSBox-X offer the following table of correspondences between processors and DOSBox cycles:

Screenshot_2023-04-16_10-59-43.png

The question about replay controls is an interesting one, which I think we might study by what happens around the divergence points. I'll also try to think of a specific experiment for that, probably involving an infinite powerslide track easy enough to be driven NoRH.

Quote from: Duplode on April 16, 2023, 04:03:45 PMThe question about replay controls is an interesting one, which I think we might study by what happens around the divergence points. I'll also try to think of a specific experiment for that, probably involving an infinite powerslide track easy enough to be driven NoRH.

For that CAS has made a useful tool.
It exports Player input from the replay.
So it is easy to see what the Player did around the divergence..

There's another one that allows you to see the keys live. Not sure if I passed that one

Quote from: Duplode on April 16, 2023, 06:05:00 AMI plan to do further analysis on these replays, as well as on the other ones shared on this thread, including running repldump on them all -- the working hypothesis being that repldump output always matches the load-from-Options outcome.

I can confirm this - I looked at Restunts' c code.
In Repldump (https://bitbucket.org/dreadnaut/restunts/src/aa1e714a66f8f9bd0d78bb1c0c3ab6b69252721d/src/restunts/repldump/repldump.c#lines-284) there is the same initialization as in Stunts (https://bitbucket.org/dreadnaut/restunts/src/aa1e714a66f8f9bd0d78bb1c0c3ab6b69252721d/src/restunts/c/restunts.c#lines-923). (check also init lines before linked ones - they are also matching)
restore_gamestate(0);
restore_gamestate(gameconfig.game_recordedframes);
With only difference being framespersec = 20; in Repldump
vs framespersec = gameconfig.game_framespersec; in Stunts

Quote from: Duplode on April 16, 2023, 06:05:00 AMAlmost all of those thirteen replays load properly from the Options menu, and diverge when played from the beginning. The sole, glorious exception is Gutix's ACT13 replay...

This quite strongly supports my second option (relax the current rule).

Quote from: Daniel3D on April 16, 2023, 05:06:34 PM

Quote from: Duplode on April 16, 2023, 04:03:45 PMThe question about replay controls is an interesting one, which I think we might study by what happens around the divergence points. I'll also try to think of a specific experiment for that, probably involving an infinite powerslide track easy enough to be driven NoRH.

For that CAS has made a useful tool.
It exports Player input from the replay.
So it is easy to see what the Player did around the divergence..

This can be seen by ordinary hex editor. And in most cases input is really not interesting. :) And in my last race - also may vary. I think I got to 1:46:00 point by slightly different inputs.

Quote from: Duplode on April 16, 2023, 04:03:45 PMAs for legacy hardware versus DOSBox on low cycles, I think what happens is that you need to run the game on very slow hardware to eliminate the divergences from normal playback, and even legacy machines are often too fast for that. For instance, the configuration menus of DOSBox-X offer the following table of correspondences between processors and DOSBox cycles:

This is like a godsend. Thank you for sharing this screenshot. I was not sure how low should I get with CPU speed with PCem.. Especially because 286 are completely different beasts to setup from 486 which I was used to. So.. I tried to create a 12Mhz 286 machine (~1510 cycles on DOSBox) and watched my replay.

Guess what - It did not crash. So the result is the same as rewind 1.05s on faster machines..
Fritest - finishes at 1:57.85
Fritest2 - Crashed into water. Went swimming with the fishes :P

These findings don't support any proposed ruling directly, only showing that at slow CPU speeds Play button behaves differently than on higher CPU speeds. At best they show that rewind 1.05s(/more in case of Fritest) is the most consistent method (and the fastest) of all of them.

Let me begin this message with some good news: testing with a broader range of example replays has allowed me to better understand how divergences interact with the replay controls, and why some replays are harder to play correctly than others. Knowing such things, in turn, should enable us to quickly identify the reachable timelines of a replay. Properly reporting those findings will require a more careful write-up, which I plan to do no later than Friday.

In the meantime, let's discuss one additional factor that hasn't been mentioned in this thread yet: cameras!

(Preliminary note: keep in mind that these results are what I'm seeing on my computer, with my usual DOSBox settings, and it's perfectly possible for you to see something else. The four example replays I'll mention are attached -- if you feel like checking them, reports of tests on different systems are always welcome!)

This tangent begins with Overdrijf's ZCT232 replay. Yesterday, while resuming an attempt to find the divergence point in it, it initially seemed I was getting different results from one day to the next. Thanks to a passing remark by @Daniel3D several days ago, I was able to quickly realise what was going on: in my system, playing from the beginning leads to a crash on the F2 camera, but not on the other ones! This replay has three main timelines:

• Crashes into the right bridge wall at 31.25;
• Crashes into the left bridge wall at 31.25; and
• Successfully jumps over the bridge and goes on to complete the lap.

I get timeline #1 by loading from options as in my proposal (or fast-forwarding the tape beyond the divergence), timeline #2 by playing (at normal speed) from the beginning on the F2 camera, and timeline #3 by playing from the beginning on the other cameras. Since I naively had been doing my tests always on F2, there was no way I'd get to see the successful timeline unless I coincidentally changed cameras.

Now, while this is a bizarre effect, one might hope that knowing the confounding factor would make things more predictable. But alas:

• "Maybe it's specifically the F2 camera which is buggy." Nope: Frieshansen's final ZCT251 replay (the one actually on the scoreboard, not one of those shared here) crashes at 1:12.85 from Options, crashes at 1:05.85 when watched from the beginning on F1, F2 or F3, and completes the lap only when watched on F4. (In contrast, Frieshanen's withdrawn replays from the same race (https://forum.stunts.hu/index.php?msg=89480) load and play normally in my system.)
• "Okay, so perhaps the F4 camera always works." Nope again: Alan's final ZCT256 replay (which AFAIK hadn't been regarded as unstable up to now) loads correctly from Options, makes a too aggressive cut at the final sector and gets 36 seconds of penalty when watched on F1, and misses the finish line altogether on the other three cameras.
• "Well, if nothing else switching cameras gives us a way of dealing with complicated cases like Overdrijf's at ZCT232." Sometimes, but not always: Alan's final ZCT244 replay (the one which wasn't deleted) sinks at 58.35 when loaded from Options, and sinks at 1:09.65 when watched from the beginning on any camera. I still haven't figured out how to make that one play to the end here. (In contrast, the deleted 1:13.80 replay (https://forum.stunts.hu/index.php?msg=82498) shows up as complete when loaded from Options.)

The conclusion I draw from these examples is that, in addition to the other issues discussed so far, the current rule doesn't actually make it easier to watch replays from the archives.

---

Quote from: Friker on April 18, 2023, 02:02:50 PM

Quote from: Duplode on April 16, 2023, 06:05:00 AMAlmost all of those thirteen replays load properly from the Options menu, and diverge when played from the beginning. The sole, glorious exception is Gutix's ACT13 replay...

This quite strongly supports my second option (relax the current rule).

Indeed. At this point, loading from Options (as in my original proposal) as the primary check and playing from the beginning (with multiple cameras if necessary) as a fallback looks like the natural thing to do. On the one hand, the load-from-Options timeline is, to the best of our knowledge, both the one consistent timeline and that most likely to complete successfully. On the other hand, playing from the beginning whenever necessary allows confirming a few more completed laps as being so, and won't significantly increase the validation workload from what it is now.

I'm watching Overdrijf's ZCT232 replay and I'm getting exactly the same as you when loading it from options and playing it with F2. Just as you describe, on different sides of the bridge at the exact same time. However, playing with F3 with the default setup from the beginning at normal speed, I see the car going near the start/finish line at 0:36.45, which triggers finishing, that is, Stunts stops reading inputs from the replay and so the car slows down to a stop almost crashing with a boat (it touches it slowly and bonces, then comes to a rest). I then tried it with the other cameras at normal speed and all yield the same result. Using double speed from the beginning, all four cameras crash at the bridge, but only F4 crashes at the beginning of the bridge. The other three crash on the right side of the bridge. I'm using 20000 cycles.

It seems to me that, when Stunts calculates the physics, strangely, instead of using a single deterministic cycle, it is using two parallel "times". That is, on one side, there's a cycle reading inputs and on another, there's a cycle executing the changes. Because they're not the same timeline, they can drift and this is more likely to happen when more things are going on inside the loop, such as rendering, but because the calculation never takes "zero" time, it can happen anytime. It'd be very interesting to get to read how the code does this, because it's really unusual. Why not use a simple single-timeline loop?  I guess it must be part of a strategy to make the game run faster on slower machines. I don't know. If this is the thing, I see no other solution but what you suggested: first, evaluate loading from Options, which is more likely to work, and if it doesn't, go and check the other ones.

Here goes the bigger post I promised on Wednesday. I have considered starting a separate thread for it, but at the end of the day it belongs here: there's a ton of relevant context in earlier replies (and it wouldn't have been possible without the input of you all). I'll eventually get to shape it into a Wiki article or two, to keep the information easily accessible.

The main goal here is showing ways to quickly find and watch reachable timelines for a replay in your system. In the process, at least one previously undocumented aspect of the replay system will be reported on.

Definitions

I'll start with a few definitions, just to avoid mixing up similar sounding things. Let's begin with the names of the replay controls, largely taken from the game manual:

(https://i.imgur.com/Vzqevw8.png)

Next, a few notions about unstable replays:

• Fast-forward timeline: the events in a replay as seen by fast-forwarding it from the beginning, loading it from the Options menu, or reproducing it with repldump. To the best of our knowledge, the fast-forward timeline is the same across systems and settings. It is this timeline my proposed verification method checks. (Note the relevant replay control is fast forward, and not fast play/"double speed".)
• Divergent timeline: a sequence of events in a replay different from the one in the fast-forward timeline. A divergent timeline might be seen by playing the replay (that is, watching with the play button), or through some other use of the replay controls. Whether it will actually be seen can depend on the computer running the game, the DOSBox cycles setting, camera choice, graphics settings, and heaven knows what else.
• Divergence point: A point in the replay at which two or more timelines diverge. Divergence points are typically (perhaps always) vertex points in a powergear slide, in which the car stops for a moment before shooting into a different direction (note the converse is presumably not true: there are many powergear slides which, as far as we know, don't lead to divergences). By convention, I will report divergence points using the last frame before the timelines split (for instance, Friker's deleted ZCT260 replay has a divergence point at 1:46.00 (https://forum.stunts.hu/index.php?msg=89481)).
• Unstable replay: A replay known to have a divergence point and multiple timelines when reproduced in some way on at least one system. (Given the way factors external to the game affect the observations, "known to be unstable" is arguably more accurate than "unstable", but let's try to keep the language straightforward.)

How rewinding works

Why are some divergences so easy to fix that you barely notice them, while others are recalcitrant enough to be nightmare fuel? The answer lies in a replay mechanic that is an integral part of RH racing: rewinding.

The first thing to note is that the game does not rewind a replay by moving backwards in time. Rather, it fast-forwards the tape from the beginning to the point you want to reach. That is why rewinding is so slow on the 1990 Broderbund (Stunts 1.0) version of the game (and gets slower the further ahead the tape is). To improve on that, later versions store checkpoints of the game state every thirty seconds, so that there is no need to fast-forward from the beginning, but only from the latest checkpoint. For a demonstration, rewind frame by frame across a xx:00 or xx:30 timestamp. You'll notice the game takes noticeably longer to update once you go behind the checkpoint:

(https://i.imgur.com/EL4SrLR.gif)

That's a nifty optimisation if replays always play in consistent ways. If they don't, though, things get interesting. For instance, if you are on a divergent timeline, and the divergence point lies behind a checkpoint, there will be a visible discontinuity when you rewind past the checkpoint. Depending on how far away checkpoint and divergence are, the discontinuity can be very obvious, as in Usrin's ZCTP03 replay around 2:00 (divergence at 1:38.95)...

(https://i.imgur.com/dCcYpyf.gif)

... or incredibly subtle, as in Overdijf's ZCT232 replay around 0:30 (divergence at 28.05):

(https://i.imgur.com/ihdVgFH.gif)

Why rewinding fixes replays (sometimes)

To keep things simple, for now let's just consider replays with a single divergence point. Suppose that you're on a divergent timeline. Since rewinding amounts to fast-forwarding from the nearest checkpoint, any rewinding within the 30-second window between checkpoints which contains the divergence point switches to the fast-forward timeline. That being so, if said window lies behind a checkpoint, you see discontinuities like the ones above upon crossing it, and rewinding without crossing it will preserve the divergence. However, if you're already into the window, any use of the rewind button will instantly snap you back into fast-forward timeline. For instance, here is what would happen were I to attempt fixing this wayward jump in the divergent timeline of my ZCT099 replay (divergence at 42.00):

(https://i.imgur.com/B0KSzY7.gif)

This explains why most unstable RH replays complete the lap successfully in the fast-forward timeline. What typically happens is that the pipsqueak enters a divergent timeline while driving and then, a few seconds later but before reaching a checkpoint, rewinds slightly to fix something. At this point, the replay switches to the fast-forward timeline (a change that might be barely noticeable, depending on how close the divergence point is), and the pipsqueak completes the lap on it. For the same reason, replays driven like that are easily played to the end: once the spectator notices something amiss, like an unexpected crash, any rewinding will put the replay back into the fast-forward timeline, which will be successful.

Instability usually gets harder to handle, though, if the divergence point is just before a checkpoint. The most familiar example is Overdijf's replay at ZCT232. Here is a demo of its successful divergent timeline:

(https://i.imgur.com/62XkjhR.gif)

My conjecture about what happened in this case (with apologies to @Overdrijf if I'm guessing it wrong) is that Overdrijf entered the divergent timeline at 28.05, completed the jump over the bridge on the first attempt and found no need to rewind past the 30.00 checkpoint for the rest of the session, until the lap was done. That being so, the lap was completed on the divergent timeline. To watch that timeline, though, you need to play the lap from 28.05 or earlier and hope for the best.

Two corollaries are worth mentioning here:

• Firstly, attempting to "fix" an unstable replay with a successful fast-forward timeline can make things worse if the successful timeline becomes a divergent one which is hard to reproduce. This is likely what happened to @Frieshansen 's laps on ZCT251.
• Secondly, any NoRH unstable replay must be completed on the divergent timeline. Since NoRH unstable replays are very unlikely to arise in practice, given how instability is associated with powerslides, I had to prepare a set piece; the resulting replay is attached to this post. You'll readily notice there's a moat surrounding the middle of the map. In the divergent timeline, which is the one I have driven NoRH, the car sinks from inside the moat at 2:09.15, whereas in the fast-forward timeline it sinks from outside the moat at 1:56.95. The divergence point comes very early at 30.50, and for a long time after that the differences between the timelines are barely noticeable by the naked eye. Below are two example screenshots at 1:20.00 (left is divergent, right is fast-forward):
  
  
  
  

  (https://i.imgur.com/ltgHUBu.png)

  (https://i.imgur.com/42xp0J2.png)

That being so, we can turn to @dreadnaut 's question from last weekend:

Quote from: dreadnaut on April 16, 2023, 11:01:14 AMI have one more question: can we generate an unstable replay without using the "forward" button of the replay controls?

Yes: divergences can happen as you drive a lap or watch a replay without anything being done with the replay controls, to the point that unstable NoRH replays are possible. Broadly speaking, the role of the replay controls can be best described as providing a way to switch timelines, not to create them.

The alternative divergent timeline

The "broadly speaking" qualifier was added to the paragraph just above due to a kind of divergent timeline which is unlikely to arise except by toying with the replay controls. It was discovered by @Argammon , with the defining example being the parked-by-the-beach ending (https://forum.stunts.hu/index.php?msg=89483) of Friker's deleted ZCT260 replay. If you advance the tape to the divergence point (in this case, 1:46.00) and play to the end, you will get a divergent timeline. However, fast-forwarding a single frame ahead from the divergence point and playing from there can result in a slightly different divergence. With Friker's replay, the margins are thin enough that such a subtle change leads to a finish completely different from what happens in the two main timelines.

(As a reminder of how divergent timelines depend on cameras, it is worth mentioning that I don't get the alternative timeline of Friker's replay on the F1 camera, only on the other ones.)

Fast play (aka double speed)

What about fast play -- is it a viable way of watching unstable replays, as has been suggested in the past? It largely is, though, not for the first time, it's a little more complicated than we might have expected.

In the examples I've seen so far, fast play follows either the fast-forward timeline or the (main) divergent one, depending on where fast play is started. Assuming a staring point earlier than the divergence, we have:

• If the timestamps of the starting point and divergence point end with the same digit, it will follow the divergent timeline.
• Otherwise, it will follow the fast-forward timeline.

For instance, fast-playing Friker's deleted ZCT260 replay from the beginning (0:00.00) leads to the (crashing) divergent timeline, as the divergence point is 1:46.00, while starting from 0:00.05 leads to the (successful) fast-forward timeline.

Fast play works by stepping through each frame of the simulation normally, as it must be, but only rendering graphics for every other frame (note how the timestamps in the replay bar advance by 0.10 s while using fast play). That we seemingly get different results from fast play depending on whether graphics get rendered at the divergence point is one more piece of evidence suggesting that divergences fundamentally have to do with graphics rendering in the game loop.

Navigating unstable replays

It's about time to try and distill practical advice from the information above:

• Playing an unstable replay from the beginning, or from any point before the divergence, will lead to a divergent timeline. What this timeline will look might depend on a host of factors, most notably your system and the chosen camera.
• Fast playing an unstable replay from either the beginning or 0:00.05, depending on where the divergence point is, will lead to the fast-forward timeline.
• Loading the replay from the Options menu and fast-forwarding the tape from the beginning to the end lead to the same result: the final state of the fast-forward timeline.
• Loading from Options and rewinding will keep you in the fast-forward timeline. (Fast-forwarding is okay as long as you don't stop at the divergence point, nor play through it.)
• The divergence point will almost certainly be at the vertex of some powerslide, when the car changes direction suddenly.
• To find the exact frame of the divergence point, load from Options (or fast-forward to the end), rewind to a candidate frame, and play from there. If you stay on the fast-forward timeline by doing so, the divergence is further back. If you switch to the divergent timeline, either you stopped exactly at the divergence point (and redoing it on the next frame will lead to the fast-forward timeline) or the divergence is further ahead.
• Watching both the fast-forward and the divergent timeline in the manner described above will usually be enough give you an upper boundary for where the divergence point can be (it must be before the timelines become visibly different). If somehow that isn't helpful, you can systematically find the 30-seconds window the divergence must be in by playing from the beginning, pausing at ~29.95, ~59.95, etc., and rewinding a single frame at each stop until you see a timeline change.
• If you see an unexpected crash while playing an unstable replay, and rewinding doesn't instantly switches timelines, chances are the divergence point is on the other side of a 30-seconds checkpoint. (A straightforward example is my ZCT232 replay: divergence point at 27.95, crashes on the divergent timeline at 33.05. Overdrijf's ZCT232 replay is similar, but more confusing still because the fast-forward timeline isn't the successful one.)

Quote from: Cas on April 22, 2023, 01:38:37 AMHowever, playing with F3 with the default setup from the beginning at normal speed, I see the car going near the start/finish line at 0:36.45, which triggers finishing, that is, Stunts stops reading inputs from the replay....

That should not happen I believe. The game stops reading input when finish is triggered and the 20 ticks are counted..

Quote from: Duplode on April 22, 2023, 10:34:40 AMMy conjecture about what happened in this case (with apologies to @Overdrijf if I'm guessing it wrong) is that Overdrijf entered the divergent timeline at 28.05, completed the jump over the bridge on the first attempt and found no need to rewind past the 30.00 checkpoint for the rest of the session, until the lap was done. That being so, the lap was completed on the divergent timeline. To watch that timeline, though, you need to play the lap from 28.05 or earlier and hope for the best.

That sounds about right. It wouldn't have been my first attempt at that section, but I'll happily believe that it was the first attempt that happened after getting the divergence. I tend to go "hah, that trick worked, awesome, next part". I might go back to have certain inputs pressed during the jump, but the 0:30.00 checkmark is pretty early in this jump, I probably just didn't go that far.

Quote from: Daniel3D on April 22, 2023, 11:00:36 AMThat should not happen I believe. The game stops reading input when finish is triggered and the 20 ticks are counted..

The 20 ticks countdown only happens while driving. If you're watching a replay on a timeline that terminates early, the tape will continue for as many frames as there were in the saved replay even after the lap is finished, in the way Cas described.

By the way, that's indeed a new timeline for me, @Cas  :o I still want to try running the game on different environments here (e.g. QEMU + DOS) to see any of the timelines being reported here that I can't access through DOSBox -- or even brand new ones!

Also, it's about time we do justice to Overdrijf's ZCT232 replay. Unlike I had been saying before, there is nothing pathological about it: just  a bit of unlucky timing (getting a divergence right before a replay checkpoint, as detailed in my previous reply) leading to the lap being completed on a divergent timeline, one which played from the beginning to the finish on his system.

I'm worried to derail the awesome work going on here ;D  Still, I think here's not really much use in the rule as-is: it's technically 'unfounded', does not match what the [visible] community sees as a priority, and a pain for pipsqueaks.

Options I can think of:

1. Keep rule as-is - Because I'm evil! ::)

2. Relax to Duplode's original proposal - If I understand correctly, if a replay loads to completion, it will also pass the "rewind just before finish line" test, so I'm not sure if this is useful

3. Relax to "replay is valid if it loads to completion" - would still exclude "load to crash" (e.g., Overdrijf's ZCT232 replay)

4. Relax to "replay is valid if it loads or plays to completion" - to casts a wider net, easily validated?

5. Return to pre-2021 rules: "replay is valid if it can be shown to complete" - would cover both fast-forward and divergent timeline replays; we could require the driver to supply the correct steps to make validation of tricky cases easier.

Any others I have missed?

Quote from: dreadnaut on April 22, 2023, 10:52:30 PMI'm worried to derail the awesome work going on here ;D

Maybe split the topic and move all research to reverse engineering?

Quote from: dreadnaut on April 22, 2023, 10:52:30 PM.Options I can think of:

• 1 | Keep rule as-is
• 2 | Relax to Duplode's original proposal
• 3 | Relax to "replay is valid if it loads to completion"
• 4 | Relax to "replay is valid if it loads or plays to completion"
• 5 |Return to pre-2021 rules: "replay is valid if it can be shown to complete"

On option 4 and 5, that is difficult,
We would need an X amount of people verifying the validity of it playing to finish.

on 2, rewind i bit and play would filter out false positives. A replay that ends on the road without crashing doesn't necessarily mean that it passed the finish. That is the usefulness of rewinding a bit.

@dreadnaut No derailment there, just a ton of switches in this stretch of railroad  :D Hand on heart, two weeks ago I didn't foresee there being so much investigation in this thread -- looking back, though, it was all for the best. Anyway, let's zigzag back to the rules. I think your options largely cover it; below are some quick notes on them.

Options #2 ("Duplode's original proposal") and #3 ("replay is valid if it loads to completion") are the same. As @Daniel3D notes, the rewinding part is just meant to catch laps that look okay but aren't (e.g. passing just wide of the finish tile, or having undeclared penalty time).

Option #4 ("replay is valid if it loads or plays to completion") is Friker's compromise: validate according to my proposal and, if it fails, try playing from the beginning, on multiple cameras if necessary. It is my favourite option at the moment. Much like you say, it casts a wider net while keeping the whole of the validation process in your hands. The one extra subtlety this option has is that, if it is adopted, pipsqueaks will only be 100% sure their replay will be deemed valid if it loads from Options -- they can send a replay which doesn't, but there's a nonzero risk of it being rejected due to you being unable to play it. That might be worth an explicit mention in the rules.

As for #5 ("replay is valid it can be shown to complete"), while I certainly like how it is the most liberal option of all, the procedural untidiness of allowing external proof (videos? testimony from other pipsqueaks?) does count against it. Though I think it could be a workable solution, at this moment #4 might offer a better balance between leniency and clarity.

(Ideally, we would figure out the technical means to easily explore all possible timelines, including the ones not directly available by running the game with one's usual setup, thus covering the external proof requirement of #5 in a fully reproducible way, but we aren't there yet.)

If it weren't that different computers and set ups can result in different time-lines, I would say first load from Options and if that fails, hand to the pipsqueak the responsibility of providing instructions that work in replaying successfully, but this might not work for everyone. I'm therefore willing to support any educated decision the community makes here and if tied, I'll just pick a choice :P

While all this research helps much more to understand how things work than to prevent or fix unstable replays, I still think it's immensely valuable and I think we should continue adding to it all we have and I agree that it things get confusing, we can just split the thread. In fact, my opinion is this should go to a wiki article as well!  There are many points here Duplode makes that I hadn't considered at all.

Adding more to the research... I had not considered these "checkpoints", although I did know about how rewinding actually fast-forwards. It looks like Duplode's work can help us see ahead when a replay could result in instability and take special care during RH to ensure it won't fail. While Duplode took a very different angle to look at the problem from the one I did, both seem to be very compatible... especially about how rendering time seems to be the primary culprit here.

I will make some predictions about things we may observe in the future if things are the way they're looking to me:
- We'll find that replays with no PG can be unstable. The faster the speed, the more likely. PG replays are more likely to be unstable only because they are so fast
- We'll confirm that turning while at high speed increases the chance of instability and that a replay may be unstable because of fast speed, yet not noticeably unstable because the car isn't turning that much, so the result is practically the same
- We'll encounter more unstable replays in the future coming from low-cycle emulation than from high-cycle emulation, although all can be, because at high-cycle, the rendering time is shorter

My understanding about what causes this in the first place is that:
- Stunts uses an asynchronous loop, like when you have a ship you can move with your keys while enemies move around. They have to keep moving even if you don't. This results in a non-deterministic physics/world, but would compensate if the math were infinitely precise
- Stunts reads the input in a loop after rendering and thus, rendering time creates a time offset from the beginning of the loop to when the action turns into effect. Reading the input means from the keyboard while recording, or from the replay file while replaying.
- While fast-forwarding does not do rendering, there may exist some small actions before input reading in the loop, so while extremely unlikely, fast-forwarding can also produce a difference from system to system. This is completely theoretical and I don't expect we'll ever see it happening
- Increments (deltas) that are added at each iteration of the loop to calculate new positions and speeds have a limited precision. The adding of these increments is the core of the creation of instability. I suspect that Stunts tries to compensate the time offset when rendering by removing time from these increments, but because the precision has a limit, information is lost, causing an error that accumulates with time. On faster driving, increments are larger and so are errors. If vector directions vary slowly and smoothly, these errors may not be as noticeable, but they exist

My recommendations would be:
- When recording a replay, use fast emulation. If you're going to drive a fast car or use PG, make it even higher or maybe reduce the rendering details
- On high speed turns, while building a RH replay, take a look at what replaying looks like and how it compares to fulling fast forwarding to have a chance to fix problems before working more on an already unstable replay

Expanding a bit on what I've said yesterday (https://forum.stunts.hu/index.php?msg=89656) about option #4, it is a peculiar rule that kind of feels like an unstable equilibrium:

• On the one hand, if everyone is aware of the details of the rule and plays it safe, ensuring replays loads as complete from Options, it effectively reduces to #3. Then again, it might be a touch too optimistic to assume full awareness of the details, and personally I still lean towards being as lenient as we reasonable can in such cases.
• On the other hand, if there are replays that don't load as complete from Options, whether those replays will be ruled as valid or not will still depend on the specifics of the admin's computer/setup. It remains a big improvement over the status quo, as a much smaller amount of replays will be affected, and there's clarity about what's going on and which replays will be at risk. Nonetheless, there might be a case for taking that load from the admin's shoulders by relaxing it all the way to #5: accept any reasonable evidence of completion, asking the pipsqueak to provide it if necessary, or setting up a procedure to get extra eyes (and computers) looking for a successful timeline.

So, while at the moment I feel like #4 is a good enough compromise, #3 and #5 are defensible options too.

---

A couple brief reports on tests with alternative setups:

• I tested the game on 86box (http://86box.net/) (a fork of the PCem emulator @Friker told us about, which I found easier to get working here) and FreeDOS, using a 100MHz 486 as my primary choice of emulated processor (the configuration file is attached for reference). The results are consistent with what we have seen so far: the fast-forward timelines remain the same, and switching to a slower processor can affect and/or remove divergent timelines. Unsurprisingly, the divergent timelines often are different from what get with DOSBox. For instance, Overdrijf's ZCT232 replay crashes at the side of a ramp at 34.15 on the F1 camera, which is what dreadnaut originally saw back in November 2020. Also, Alan's final ZCT244 replay, which doesn't play to end at all on DOSBox for me, successfully plays to the end on F2.
• On DOSBox, changing the emulated video device (and the corresponding graphics driver in Stunts' SETUP) also affects divergent timelines. For instance, with CGA, Hercules or Tandy graphics Overdrijf's ZCT232 replay crashes on the left bridge wall at 31.25 when played on F1, instead of completing the lap as on VGA. Similar things also happen on 86box.

---

@Cas I believe the only thing that might be reasonable to ask drivers to do is ensuring replays load as complete from Options (which would be mandatory with option #3, and would remove any risk of disqualification with #4). Requiring changes to DOSBox or Stunts settings makes the game less accessible, and is not guaranteed to help.

@Duplode, of course I wouldn't request pipsqueaks to have to go through so much struggle or make them responsible for so many safety measures. These are recommendations in case you're worried that your replay may be unstable and you feel you want to make sure you're not working on something that will later bring problems. Something for people to do when they want it, not as a request.

For instance, I imagine maybe some of us have DOSBox configured by default at low cycles and changing this is a very simple thing you have to only do once and might help, so why not?  Not that it is required.

I bet for if the replay can be run with F1, F2, F3 or F4 view then accept it. 100%
Then we could talk about dosbox simulation and number of cycles too.

I vote for proposal 4. Accept a replay if it loads correctly from options or plays correctly with any camera view using normal and/or double speed. For example, if the replay plays correctly only with the F2 camera using double speed that is ok.

Proposal 5 is what we should aim for in the future. However, I feel more research is necessary to find a way that the verification process is feasible, safe and cannot be exploited.

My heart is still in an option 5 place, but considering the administrative arguments I think option 4 has a good case for it too. I would like as many replays as possible to be valid. I don't think anyone here can really purposefully and meaningfully abuse this effect to get faster times (the example given earlier seems to be a way to get back on track after a calculation error caused a crash, the actual intended timeline is the one in which you didn't crash, and you might not have crashed using a different PC or other settings) and even if they can, isn't that just a new frontier of replay handling skills?

If we have to choose one single way of verifying (I don't really see why, but if we have to) the revelations in this topic make me say that I prefer the loading/fast forwarding method over the replay method, since it seems to be the correctly calculated line, it's more likely to match what the player drove and it always seems to have a stable result, while which branch the regular play function takes can be unpredictable.

I can respect any choice made of course, and I will support @dreadnaut's right as the main competition administrator to draw whichever line he sees fit.

I think option 5 makes sense if the "steps" are simple, straightforward, and work for everyone, such as "run it at double speed" or "play it with F2 up to 1:35 and then with F4" as more complicated example, but not if verification depends on a video or only works for some. Problem is you may not be able to tell if a procedure works for all, even after you've tried and it worked for, say, 5 people so far. For this reason, I'm more inclined towards 4.

However, same as Overdrijf says, and like I mentioned earlier, I'll support what the community decides and I too think Dreadnaut has the primary vote here.

I suggest that we (or better dreadnaut/duplode) make a decision soon. For all I know, ZCT262 allows for power-gear laps. I have no idea whether the power-gear laps are actually faster (including bonuses) than ordinary laps, but it is possible. So, it would be nice to get this settled soon.

I made sure my ZCT261 replay played to the end on my system but was a bit nerveous nonetheless because I was not 100% sure it would also do so on dreadnaut's.  :)

I am all for option 4 (maybe with a check of three independent persons).

Due to recent findings I would strongly prefer to add that a replay plays correctly from 0:00.00 or 0:00.05 is valid.

Option 5 seems to me pretty vague - I would be OK if it passes but only with the independent check.

Quote from: Duplode on April 22, 2023, 10:34:40 AMHere goes the bigger post I promised on Wednesday.
...

Duplode, thank you for your insights. I would like to reply to it but I am quietly waiting for a rule change. :) Then we can continue with a discussion about these findings.

Option #4 does seem an acceptable tradeoff between accepting as many replay as possible, and having a repeatable process to validate replays. I don't think there is a problem with "works on my machine": differences in hardware/cycles create (or collapse) the split between fast-forward vs divergent timelines.

Option #4 would accept all replays with a single type of 'timeline split'. A rejected replay would require part normal-speed play, part fast-forward, which would be more complex to identify and validate for all involved.

Rules would change from:

QuoteA replay must be a complete lap of the circuit, and should play correctly when loaded in Stunts, from "Options → Load replay". Please verify your replay before uploading it, as unplayable replays will be disqualified.

to:

QuoteA replay must be a complete lap of the circuit. Please validate your replay before uploading it, starting from "Options → Load replay". A replay is valid if it shows a non-crash frame when loaded, or if it plays from start to finish at normal speed. Invalid replays will be disqualified.

Quote from: dreadnaut on May 08, 2023, 11:43:50 AMA replay must be a complete lap of the circuit. Please validate your replay before uploading it, starting from "Options → Load replay". A replay is valid if it shows a non-crash frame when loaded, or if it plays from start to finish at normal speed. Invalid replays will be disqualified.

[/quote]

I'm sorry I can't tell what changed. :-\ For example, this new rule would change something in the previous results if it were implemented before?

Yes, they all load not crashed if I'm not mistaken.
It would be good practice to rewind a bit to check if the finish line has been crossed.
But this is the most player friendly version of the rule as can be (in my opinion)

Very good then! ;D

@-dreadnaut While I also find #4 an acceptable tradeoff, it's worth noting it might be slightly harsher than that. More specifically:

Quote from: dreadnaut on May 08, 2023, 11:43:50 AMI don't think there is a problem with "works on my machine": differences in hardware/cycles create (or collapse) the split between fast-forward vs divergent timelines.

There are (at least) two different manifestations of "works on my machine":

• Replays which complete successfully on the fast-forward timeline and diverge on some machines but not on others. Frieshansen's withdrawn ZCT251 laps are an example. These will be accepted by #4.
• Replays which don't complete in fast-forward, and which diverges on different ways depending on the machine. Examples include Overdrijf's ZCT232 replay (plays okay for me, didn't for you) and Alan's final, 1:14.15, ZCT244 replay (played okay for you, doesn't for me). These still are at risk under #4.

The new wording of the rule looks fine 👍 Is it worth it to mention playing with multiple cameras, or would that be excessive detail for that page? (Edit: another plausible reason not to mention it is that, even if validation comes down to cameras, the pipsqueak will see the lap being complete on F1, and so changing cameras isn't actionable advice for them.)

@Friker Sure! Be it in this thread or in another, we'll carry on the investigation :) By the way, I still need to condense the information here fot the Wiki.

Quote from: Daniel3D on May 08, 2023, 04:46:54 PMYes, they all load not crashed if I'm not mistaken.
It would be good practice to rewind a bit to check if the finish line has been crossed.

Some replays show a cracked windshield when loaded, but play correctly from the start. Crossing the line is covered by the first sentence: "A replay must be a complete lap of the circuit."

Quote from: Duplode on May 08, 2023, 10:47:00 PMIs it worth it to mention playing with multiple cameras, or would that be excessive detail for that page?

Switching to different camera would be "custom instructions", and fall under #5. Option #4 was meant to cover only replays that play at normal speed with the default camera.

Quote from: dreadnaut on May 09, 2023, 01:03:55 AMSwitching to different camera would be "custom instructions", and fall under #5. Option #4 was meant to cover only replays that play at normal speed with the default camera.

Fair enough. While this would have ruled out Frieshanen's final ZCT251 replay, he wouldn't have needed to fix it in the first place, as the withdrawn laps are complete on the fast-forward timeline. The hardest part might well be not switching to F2 on instinct when validation time comes  :)

Quote from: Duplode on April 22, 2023, 10:34:40 AMFast playing an unstable replay from either the beginning or 0:00.05, depending on where the divergence point is, will lead to the fast-forward timeline.

@Duplode , please, confirm that this happens only with fast-forwarding.

Quote from: dreadnaut on May 08, 2023, 11:43:50 AMA replay must be a complete lap of the circuit. Please validate your replay before uploading it, starting from "Options → Load replay". A replay is valid if it shows a non-crash frame when loaded, or if it plays from start to finish at normal speed. Invalid replays will be disqualified.

I would still add something about crossing a finish line (extended to the whole block). But maybe it is obvious for others.. Then a question is - what is a real finish time? - because @Argammon showed very bizarre things. :)

Quote from: Friker on May 09, 2023, 10:12:42 AM@Duplode , please, confirm that this happens only with fast-forwarding.

It's fast play (also known as double speed) rather than fast forward, but other than that, yes indeed. Fast play doesn't actually make a difference when it comes to validation: depending on the starting frame, you'll get either the fast-forward timeline or the "main" divergent one, which are the ones checked by #4 anyway. (In this context, fast play is mostly useful for providing a way to watch the successful timeline without pausing.)

Quote from: Friker on May 09, 2023, 10:12:42 AMThen a question is - what is a real finish time? - because @Argammon showed very bizarre things. :)

I belive the sensible thing to do is to keep using the file length-based time shown by the site, which that corresponds to what the pipsqueak got while driving. The only tricky case, which should be very rare, is if the lap completes successfully in the main timelines, but neither matches the file length time. Options in that case include sticking with the slowest time, and ruling the replay invalid.

Quote from: Duplode on May 09, 2023, 01:03:25 PM

Quote from: Friker on May 09, 2023, 10:12:42 AM@Duplode , please, confirm that this happens only with fast-forwarding.

It's fast play (also known as double speed) rather than fast forward, but other than that, yes indeed. Fast play doesn't actually make a difference when it comes to validation: depending on the starting frame, you'll get either the fast-forward timeline or the "main" divergent one, which are the ones checked by #4 anyway. (In this context, fast play is mostly useful for providing a way to watch the successful timeline without pausing.)

You are right, I had fast-playing in mind.

Quote from: Duplode on May 09, 2023, 01:03:25 PM

Quote from: Friker on May 09, 2023, 10:12:42 AMThen a question is - what is a real finish time? - because @Argammon showed very bizarre things. :)

I belive the sensible thing to do is to keep using the file length-based time shown by the site, which that corresponds to what the pipsqueak got while driving. The only tricky case, which should be very rare, is if the lap completes successfully in the main timelines, but neither matches the file length time. Options in that case include sticking with the slowest time, and ruling the replay invalid.

One can argue that he drove with fastest time in mind (even if that does not make sense because then a replay would finish sooner..) that's why I would like that clarified. :) I am also inclining to the file length/slowest time.

Quote from: Duplode on May 09, 2023, 01:03:25 PMThe only tricky case, which should be very rare, is if the lap completes successfully in the main timelines, but neither matches the file length time.

In such cases you could rewind to just before the finish line and continue from there. (Right?)

Quote from: Daniel3D on May 09, 2023, 07:19:34 PMIn such cases you could rewind to just before the finish line and continue from there. (Right?)

In the unlikely situation I'm thinking of, it wouldn't. For instance, consider a replay which is completed by the pipsqueak on the divergent timeline, crashes on the fast-forward timeline, and diverges in a different way on dreadnaut's machine so that it clips the finish tile and ends prematurely with penalty. In that case, the only laptime dreadnaut will see isn't the file length one. Outside of perfect storm cases like that, though, the file length lap time will be good enough.

Whops, I forgot about this thread :o

I have one question remaining: when should the updated rule (https://forum.stunts.hu/index.php?msg=89790) come into play?

A) ZCT262 - This race, I'm hiding a really tricky powergear lap!
B) ZCT263 - Next race, this is urgent
C) Season 2024 / ZCT270 - No in-flight patching

I'd rather not (A), but let's see what you think!

Quote from: dreadnaut on May 30, 2023, 12:02:21 PMI have one question remaining: when should the updated rule (https://forum.stunts.hu/index.php?msg=89790) come into play?

I'd be happiest with (A), but (B) is an acceptable substitute if you'd rather stick to the general principle of not changing rules mid-race. There's no point in waiting until 2024 IMO.

I'm alright with all options!

As I mentioned previously, I'm happy with the current rule, so I don't have any preference when the updated rule comes into play.

Quote from: Duplode on May 30, 2023, 05:29:00 PM

Quote from: dreadnaut on May 30, 2023, 12:02:21 PMI have one question remaining: when should the updated rule (https://forum.stunts.hu/index.php?msg=89790) come into play?

I'd be happiest with (A), but (B) is an acceptable substitute if you'd rather stick to the general principle of not changing rules mid-race. There's no point in waiting until 2024 IMO.

Same here!

Quote from: alanrotoi on May 31, 2023, 04:52:59 AM

Quote from: Duplode on May 30, 2023, 05:29:00 PM

Quote from: dreadnaut on May 30, 2023, 12:02:21 PMI have one question remaining: when should the updated rule (https://forum.stunts.hu/index.php?msg=89790) come into play?

I'd be happiest with (A), but (B) is an acceptable substitute if you'd rather stick to the general principle of not changing rules mid-race. There's no point in waiting until 2024 IMO.

Same here!

Same

Quote from: Duplode on May 30, 2023, 05:29:00 PMI'd be happiest with (A), but (B) is an acceptable substitute if you'd rather stick to the general principle of not changing rules mid-race. There's no point in waiting until 2024 IMO.

There is also a principle of not changing rules mid-season — even when the replay validation was introduced, it only came months later if I remember correctly — but (B) seems the best balance, I'll update the rules on Sunday 👍

@dreadnaut All right, thanks 👍

Quote from: dreadnaut on May 30, 2023, 12:02:21 PMI have one question remaining: when should the updated rule (https://forum.stunts.hu/index.php?msg=89790) come into play?

Just a quick reminder before the ZCT264 update  :)

Quote from: Duplode on July 01, 2023, 09:01:51 PMJust a quick reminder before the ZCT264 update  :)

Very good idea! Rules (https://zak.stunts.hu/rules/2023) updated :)

QuoteThe System

• ...
• Rule change, applies from 2023-07-02
   
  • A replay must be a complete lap of the circuit, and should play correctly when loaded in Stunts, from "Options → Load replay". Please verify your replay before uploading it, as unplayable replays will be disqualified.
     
  • A replay must be a complete lap of the circuit. Please validate your replay before uploading it, starting from "Options → Load replay". A replay is valid if it shows a non-crash frame when loaded, or if it plays from start to finish at normal speed. Invalid replays will be disqualified.
  
  
• ...

And so the story arc of one of the wildest threads in recent memory is wrapped up. Thanks to everyone who contributed, and to @dreadnaut for bearing with us! Special shout-out to @Friker and @Frieshansen , who brought the decisive experiments to the table. I'm well behind schedule when it comes to planned Wiki article updates, but I'll get around to reporting the results over there!

No worries, thank you all for the thorough investigation of replaying, and for all the feedback and suggestions :D