ReplayHandler v.0.1

[Duplode]

What is it about: since I was willing to exercise my elementary Java skills a bit further, I decided to start a small project and make a new, Java-based, implementation of RPLInfo. While the relevance of coding yet another replay checker is debatable (especially a console based one, as learning how to design decent user interfaces is not on my short-term plans...), what I am really interested in is to use my implementation as back-end for batch file processing, thus creating helper functions useful to competition managers and replay archive organizers. For instance, the first utility I'm working on is a .HIG generator: the program parses the .RPL files, sorts them according to replay times and then print the data into binary .HIG format, using the file name for the pipsqueak name field. I think that would be a cool addition in future replay packs, as well as helping to keep record of race results for the posterity. Other possibilities that can be envisioned include an offline management system for competitions without php sites, which would get aptly-named replay files from a new replays folder, do the proper validity checks (track, car, lap completion), add the new laps to the scoreboard (eventually replacing older laps from the same pipsqueak) and print the updated scoreboard to a .txt . If you can think of other useful stuff that could be added, please do make the suggestions!  I will publish a preliminary version of the code for you to test as soon as it gets a bit more clean and usable.

[zaqrack]

Sounds good for a start!

I'll just let my thoughts roam free. The following might be too wild or simply stupid ideas:

Did we ever make experiments, on how writing some extra bytes to a replay modifies it?
If it wouldn't interfere with the replay itself, we could use this tool's next generation to write competition, track, pipsqueak name into the file, even along with some short comments or other identification. This could be then the new generation RPL, and online competitions would require replays edited (or rather only parsed) through this tool. As it will be in Java, it'd be simple to use online, with a background database handling the competition names, track names, etc. Competition replays can be also validity-checked with trk files. 
We could also create a replay sending form under stunts.hu, with proper login needed. pipsqueak name is auto loaded then, voila. All the other required extra data could be selected from dropdown lists, and the java tool would send the replay to the manager of the selected competition in mail (or in case of zakstunts, in simple php variables) along with the extracted data and information in mail body.

This would also guarantee, that there would be no lost or unknown replays on our hard disks in the future, loading them in this tool would display all the additional data we need to identify them. 
 

[Mark L. Rivers]

What I'm reading is music for my ears... 
Having not a PHP host, I'm looking forward to see how the new tool could help me in order to make more automatic certain part of the ordinary job. This could mean SDR surviving in the future... 

[BonzaiJoe]

That sounds really great! I'm looking forward to having these options. Particularly the .hig generator!

[Duplode]

Did we ever make experiments, on how writing some extra bytes to a replay modifies it?
If it wouldn't interfere with the replay itself, we could use this tool's next generation to write competition, track, pipsqueak name into the file, even along with some short comments or other identification. This could be then the new generation RPL, and online competitions would require replays edited (or rather only parsed) through this tool.

I performed two tests with extending default formats yesterday:

• Writing more than seven entries to a .HIG, hoping that they would be stored normally, even if not displayed in game. It seems one can put up to 12 entries in a .HIG. More than that, however, will make the .HIG overflow into the track file within the game RAM, causing gibberish tiles to be added to the bottom of the track and the .HIG not getting loaded properly. That means we have to be very cautious with unforeseen consequences of appending stuff to the default formats... and since most likely the game will rewrite the scoreboard as soon as you get a new replay and reset it to 7 entries, the best way to store the full results would be generating a new file for each set of 7 pipsqueaks (the main one being, say, ZCT86.HIG and the others ZCT86.HI2, ZCT86.HI3, etc.)
• Appending a zero-terminated tagging string "SIGNATURE" to the end of an .RPL followed by the zero-terminated pipsqueak name, so that Zak's suggestion could be implemented. It seems that, unlike in the .HIG case, there are no visible problems generated by the file overflowing its reserved memory area, so it may well be a viable strategy. Of course, adding such signatures would break all existing replay time identification algorithms based on file length, but that's a workable obstacle I guess (and naturally in case we do use signatures the new replay checking tool should be able to handle both signed and non-signed replays).

The bottom line is that the programmer should exercise a lot of caution when extending default file formats. But it works...

As for a full-fledged competition management system, there are two main requests that need to be fulfilled:

• Creating a round management system that is pipsqueak-based, as opposed to replay-based as my simple test version is. The coding itself should not be too much of a problem (make a Pipsqueak class which stores a Replay object and make the Pipsqueak able to update his Replay accordingly); the main issue is finding a convenient mechanism for automatically (i.e. without the manager having to enter the names manually) finding to whom does a replay belong. File-name based, directory-path based, RPL-signature based... I don't know, and accept suggestions on what would be more user-friendly.
• Implementing Rounds of different kinds for different competitions, so that stuff such as replay name parsing schemes and replay sorting methods can be tuned at will. I implemented it in a small scale into my test app and it works fine.

The first functional alpha version of the application is ready, and I would have posted it already weren't for a stupid little thing I couldn't figure out yet (how to make the program read an auxiliary .txt file included within the .jar package I will post...). As soon as I solve it you'll be able to test the program.

[Krys TOFF]

Sounds interesting, especially the .hig generator as BJ said. But that won't change anythig for FTT competition as I need to verify replays and videos according to noRH rules. Same apply for no-shortcuts competitions. But for other competitions this would be great.

Having not a PHP host, I'm looking forward to see how the new tool could help me in order to make more automatic certain part of the ordinary job. This could mean SDR surviving in the future... 

Change host. Php automatically updated scoreboard is website admin dream come true. I don't have anything to do except adding points to pipsqueak's database at the end of each track (and enter news and new track in database too, but that's not really long).
When I compare this to the huge amount of time that WRC Stunts and IMSA Cup required to do daily updates, phew, hail to php !

[Mark L. Rivers]

Not the main problem... The real problem is the time to learn PHP language... 

[Krys TOFF]

Mmmm, not so long if you already know before xhtml. I started with MS Word docs converted in html, then wysiwyg web editors, then I started to learn xhtml.
Then, once having a decent xhtml basis, I started to learn php and it was much faster and easier to understand than I thought.
All I learnt was from this website and its online xhtml and php/css very well done tutorials : http://www.siteduzero.com Website is in French only, but they plan to do an English version sooner or later (probably later than sooner sadly).

[Duplode]

OK, stupid problem with file paths solved, here you have the first functional alpha version. General instructions:

• Naturally, you need to have the Java Runtime Environment installed to run it. To run the program, unzip the package to some folder, reach the folder in your system's command prompt and type java -jar ReplayHandler.jar. It is possible that in Windows you can launch the .jar with a double-click, but I haven't tried it myself.
• Once in the main menu, you'll find two basic operation modes.
  
  
  
  • The RPLInfo mode will get data from a .RPL file you provide, perform the basic checks and print replay information to the screen. If the .TRK of the replay is available in the same folder of the .RPL, it will perform track checking as well. You will also have the option to test the novel .RPL signature mechanism, which will create a copy of the replay and append the pipsqueak name (assumed to be the file name for the moment) to it. 
  • The Round manager mode will get a folder name and generate a 7 pipsqueak maximum .HIG file based on the .RPLs on the folder and print some basic info to the screen. It also asks for a .TRK file path in case there is no .TRK on the folder, so it can get the proper name for the .HIG, so it is advisable you place the .TRK in the test folder. The program will overwrite any existing .HIG for the track without asking for confirmation.
• You will also see there are Zak'08 variants of the operation mode on the menu. Those will use car bonus data to print corrected times (RPLInfo) or to compare replays (Round manager). For these to work, a carbonus.dat file must be available at the folder where the .RPL files are. A sample file is provided in the .zip, with data from Z86 (Z86 replays are available too, for a quick sanity check).
• If you're launching the program from the command line, you can pass a file or directory path as command line argument. Note that the different modes will use the path as is, so if you pass a file to Round Manager or a folder to RPLInfo mode it won't work...

That's it - test, report, enjoy 

Edit: replaced the (original) bugged build for one that should work.

[zaqrack]

java class can not be loaded: java.lang.UnsupportedClassVersionError: duplode/stunts/ConsoleMenu

[Duplode]

I should check my Linux Java JRE/JDK Setup...  Anyway, rebuilt the .jar with Java 5.0 compliance. I tested it on my Windows, so this time there should be no problems... (new .zip attached on the original post)

[dstien]

It runs fine here with 1.6.0.10. It throws an uncaught exception when trying to open a directory as a .RPL file. Sorry.

[Duplode]

It throws an uncaught exception when trying to open a directory as a .RPL file. Sorry.

Thanks for calling my attention to it. That's really not surprising, as I had found that I actually was obliged to do something with the exceptions the night before! The next release will hopefully be a bit cleaner when facing such issues.

---

Currently I am beginning to study how to tackle more seriously the concept of off-line round management. As mentioned before, the limiting task for that from an user/contest manager perspective is how to assign a replay to a pipsqueak as unambiguously as possible. The pipsqueaks themselves would be defined as entities whose information can be stored in some simple database outside the program - that way, managers would only need to make use of addNewRäcer function once in the season, when the new pipsqueak takes part for the first time, and statistic-tracking and season standing computation schemes could eventually be implemented. Within the program, pipsqueaks will be mapped to their most recent replays for all purposes due. The question is by which means should we let the program perform the assignment? I can see a few basic alternatives:

• Filename based, pipsqueak-side: Competition manager will ask pipsqueaks to kindly use a standard file naming scheme when sending their replays sot that they are unambiguously assigned. That is the cleanest solution to implement IMHO, but of course it depends on the clear definition of a standard scheme and on the cooperation of the participating pipsqueaks...
• Filename based, manager-side: The obvious alternative is having the manager to rename the files as they are received. I feel (and Mark probably agrees) that doing so would defeat the purpose of the system, however... unless the contest had some sort of server-side scripting which could do the renaming as soon as the replay got submitted (did anyone say ZakStunts? )
• Directory based, manager-side: The folder storing the round replays would be a collection of subfolders, one named after each pipsqueak, from which the program would get the replays accordingly. A manager without php scripting would have to worry with downloading each replay to its proper folder. That is, however, much better than having to rename the replays, and does not rely on pipsqueak-side cooperation in any way. So that seems to be a good compromise solution.
• RPL signature based: In the lines of the advanced replay format Zak described. This solution has the advantage the extra information fully portable by storing it with the replay, and has great potential for archiving purposes (say, a Competition Archive vol. 3), specially if we can prove one can put a lot of bytes into the file without bizarre side-effects. However, it may be not so good for day-to-day usage by pipsqueaks and managers. Requiring the pipsqueaks to use an external tool to parse the .RPL could easily be an obstacle, and having the manager to do it is no better than renaming the replays. And additionally, there is the additional inconvenient of tampering with the file format: for starters, the parsed replays would not be compatible with other existing RPLInfo implementations...

Of course we could have multiple implementations, but I would really appreciate input from the managers on what seems to be the best approach in order to decide on which I should focus first. In case we should go for a filename based mechanism, the topic of file name standardization will inevitably show up, but that's better left for a future post...

[Mark L. Rivers]

• Directory based, manager-side: The folder storing the round replays would be a collection of subfolders, one named after each pipsqueak, from which the program would get the replays accordingly. A manager without php scripting would have to worry with downloading each replay to its proper folder. That is, however, much better than having to rename the replays, and does not rely on pipsqueak-side cooperation in any way. So that seems to be a good compromise solution.

Sure the best for me...