Stunts code deep dig.

[Cas]

I really have barely looked at the code at this point. This last month, I've been very busy and I'll be getting even more busy in the following weeks, so I really can't judge what has been done, even if I wanted to. I can only post about what comes to my mind and the things I imagine that I can do, but of course, I can't do anything in the end without first really digging in the code and seeing what has been done and how and what is pending, etc. To be honest, I don't know when I will be able to really sit down and do that... or rather, move my chair to my desktop computer and do that, because I've been spending the whole month sitting in front of my employer's laptop, ha, ha.

While I am a fan of FASM and I have never used another assembler to program (although I have read code in other assembly formats before), I don't really think there'll be a problem other than I will have to first learn the structural directives and try assembling the whole thing a couple of times to make sure I understood the thing well before touching anything. Believe me I don't want to skip any comprehensive analysis that has already been made. That's the most important part of what has been done!  (and what takes the longest). Interpretation of variables and functions, where they are and what they do is precisely the single most fundamental part of the work and what I would like is to make sure this is complete in the parts of the code I'm going to touch, to prevent things like what llm describes. If any unlabeled offset or function call is found, I can't touch that until I know what that is. But if a function makes reference to local variables, I don't need to know what they are if I'm modifying a function somewhere else in the code. That's what I mean. So, looking at what has been done, I would be able to surgically unplug and plug something in the place of a single routine.

About the C part, this is another part of the project. It's very good that it's being done, but I really can't help myself from it at this point. But I see that the original assembly code is there nice and clean, separate from the C, so it shouldn't be any problem if I just work on the assembly. That's what I understand.

It's very difficult to define norms when a work is non-commercial, ha, ha. We really can do whatever we want. Programming has always been a hobby for me and, since I work alone, I don't try to follow any trend. I just do the things the way that I feel is more convenient, comfortable and elegant to my own taste. I could as well just call what I do my "norms". Of course, I do not intend to do anything on the code that will stay there and later, bring a hell to you guys. I would just use the code to do something completely separate from the project. In fact, I have been considering the option of doing the "plugging" in memory instead of compiling it... that is... a TSR. But I'm not convinced about that.

Slightly off-topic
On that, I've been looking for a good way to be able to make graphical programs in C and I'm kind of stuck. People quickly throw 3rd party libraries at me and they are usually unnecessarily large and complex, with external dependencies and requiring object oriented programming to use them. There is also the option of going down to XLib, but that would make my programs GNU/Linux-only. I would like to investigate the code of FBFGX, the library that comes with FreeBasic. It is portable, so if I understand how the core of the library works, I could make my own tiny portable core and work on top of that. I really only need something that can take a memory buffer to a graphical window. I can write the primitives myself. Anything much bigger than that is too much.

[llm]

On that, I've been looking for a good way to be able to make graphical programs in C and I'm kind of stuck. People quickly throw 3rd party libraries at me and they are usually unnecessarily large and complex, with external dependencies and requiring object oriented programming to use them. There is also the option of going down to XLib, but that would make my programs GNU/Linux-only. I would like to investigate the code of FBFGX, the library that comes with FreeBasic. It is portable, so if I understand how the core of the library works, I could make my own tiny portable core and work on top of that. I really only need something that can take a memory buffer to a graphical window. I can write the primitives myself. Anything much bigger than that is too much.

1. use SDL1 or SDL2 (https://en.wikipedia.org/wiki/Simple_DirectMedia_Layer) - its C code - no OOP - it is in use absolutely everywhere - a ton of tutorials around the internet (even for FreeBasic)
2. do not fall into the not-invented-here syndrom when 3rparties are needed - they are not that evil and also in no way reponsible for any complexity in "your" code
3. wrap the code in free basic like function names, put the stuff into a different compile unit and thats it - nothing to argue
4. every idea of writing a small, fast and good portable lib yourself will just fail ultimately, thats the reason for SDLs existence
5. don't distract yourself fixing everything "around" your target goal
6. this is the way it works with everything except FreeBasic - assimilate yourself
7. im in businness for +30years, written some million lines of code and never get in any trouble using 3rd-party - so whats the point
8. prevent them if possible, but don't fight them for unrelevant or emotional concerns

[llm]

About the C part, this is another part of the project. It's very good that it's being done, but I really can't help myself from it at this point. But I see that the original assembly code is there nice and clean, separate from the C, so it shouldn't be any problem if I just work on the assembly. That's what I understand.

that means 5-10 times more bug prone, non portable assembler lines of code because you don't want to read/write C code? hard to understand

[Daniel3D]

I don't think it matters much if we work with assembly or the c code.
But I thought the game was not completely ported to c yet. And therefore we couldn't change in the c code anything until the game was fully ported.

My main goal with this project is understanding the way that the game is designed. Reading this Inc or assembly does not matter I think.

Changes I'm proposing to make are very small. And as a way to learn more. I'm hoping that this will help in creating a better port to c.
Then real changes can be made.

Of course, my understanding of the matter is rudimentary at best, so I could be wrong.
We did get a nice Ferrari edition this way though, and I am really happy with that.

[llm]

I don't think it matters much if we work with assembly or the c code.

only a non-developer can write such a sentence without being embarrassed at all

do you know that assembler is at least 5 times more code then the equivalent C code? and the possiblity to make errors is much larger

But I thought the game was not completely ported to c yet. And therefore we couldn't change in the c code anything until the game was fully ported.

when your team understands how restunts does the C linking it is possible to introduce your new code as C code - also combined with the asmorig source
but that means you all need to understand and read that stuff - the docs etc. - decide to do it the one or other way only based on the current knowledge is
not very sensfull - and there is a need to learn much if you try to work on a reverse engineering project - its just much much more harder then C or normal/typical written assembler
and you need to become familiar with the tool-chain also - the assembler, linker etc. - it is not just a simple DOS program based on assembler, its an reversed DOS program with many more constraints/problems as a typical assembler code would have

[llm]

But if a function makes reference to local variables, I don't need to know what they are if I'm modifying a function somewhere else in the code. That's what I mean.

that works only if you add the new code at the end of the binary - or else you're changing - because of the different function size when doing changes , the binary layout behind, resulting in damaging all non-symbolic non-relativ offsets after that changed function - and you will then hardly understand why you getting randomly crashe etc.
and that can even happen if you make +- 1 byte changes

so first thing would be to find out if there are still evil offsets like that, i don't know

and that is something that does not happen at all in a typical/normal written assembler program that didn't come from reverse engineering

[Daniel3D]

only a non-developer can write such a sentence without being embarrassed at all

Guilty. But it has led to beautiful things in the past.

do you know that assembler is at least 5 times more code then the equivalent C code? and the possiblity to make errors is much larger

I do know. And I am aware of the limitations.
That's why we haven't made any alterations to the source so far besides directly in the binary so we would not risk any offset.

[Daniel3D]

so first thing would be to find out if there are still evil offsets like that, i don't know

I am curious about that one as well.

I am wondering. And I am well aware that;

only a non-developer can write such a sentence without being embarrassed at all

Can it be that (because everything besides the game engine is written in c) there is no evil offset in the code and that the parts that were written directly in assembly are well contained?
I realize that when the game was developed the compiler would manage the offsets when it was recompiled after a change.
So a change in the originally c coded part can still influence the assembly written engine. But that is something I would like to test.

I'm still talking about minimal changes. But I hope that it will inspire people into joining the restunts project and help in the full port to c so that one day we can really upgrade the game. (hopefully including better graphics)

[llm]

Can it be that (because everything besides the game engine is written in c) there is no evil offset in the code and that the parts that were written directly in assembly are well contained?
I realize that when the game was developed the compiler would manage the offsets when it was recompiled after a change.
So a change in the originally c coded part can still influence the assembly written engine. But that is something I would like to test.

it "could" be that, because of the very old MSC 1.5 compiler that was use building stunts, there isn't much optimization (like generated by todays compiler) in the C based executable code parts - and the original handwritten assembler would not use dirty tricks (it think) - so yes you can be right with your assumptions

[Cas]

Well... there's one thing I am not at all good at for sure and that's making my point clear with only a few lines. I always feel like I'm not being clear enough and write more and more and then people reach wrong conclusions. I'm going to try. It won't be very brief, but at least, orderly and each point synthetic:

- Yes, I'm aware that moving the code can break it if there are references to the code itself that have not been accounted for. I think I have mentioned this problem early in this thread.
- I do think Daniël's guess is probably right, which is not the same thing as assuming that to do my work. I think he doesn't assume that either. Because the code was originally C (mostly), likely it doesn't have (many) tricky references to itself
- Your work is very professional, no doubt. Mine is not. This is not my profession. It's my lifelong hobby. That's why I do it my way. I won't force anybody else to do it my way. My main objective is not to just complete the program, but to enjoy making it. If I can't enjoy making it, I don't do it. This is not meant to be rude, just synthetic and clear. I hope you understand.
- "The way it works" is relative. It's the way some people do it. They may be many, but that doesn't mean the rest is wrong. When the first few people thought the Earth was round, most people were wrong. That happens all the time. Also, there's no need for one of the "two groups" to be wrong. They may both be right.
- What do you mean when you say SDL was made because every attempt I would make at creating a portable library would fail?  Ha, ha. Anyway, what I want is so, so, so much smaller than SDL that I can easily do it. The only reason I haven't is I don't know how because I don't know the protocols and they are not very well documented. I can find out, but now I'm so busy!
- I don't think SDL is bad. If it's non-OOP code in C, that sounds good to me.... as long as I can statically link it in my programs. But again, it's too big. It wouldn't make sense for small projects. There is one thing I don't know: whether I have to include the whole library source code with my program source code if I statically link it to it (to follow the GPL). Do you know that?
- There are good reasons for choosing to work on the assembly code instead of the C code depending on what you want to do. Mainly, that the C code is not complete yet and that, while the assembly is a very faithful representation of the original binary code, the C code is an interpretation of it, albeit a very good one. Sometimes, you will definitely want the C code, but many times, you will want the opposite for these reasons. It depends on what you want to do.
- Portability is not a concern at this point, in my opinion, since this is real mode code. Only if you get to the point to completely abstract the code from the memory structure, will you be able to consider portability... unless you're thinking of a specialised emulator.

[llm]

- Yes, I'm aware that moving the code can break it if there are references to the code itself that have not been accounted for.
I think I have mentioned this problem early in this thread.
I do think Daniël's guess is probably right, which is not the same thing as assuming that to do my work. I think he doesn't assume that either. Because the code was originally C (mostly), likely it doesn't have (many) tricky references to itself

or there a just offsets that are not symbolized in the reverse engineering process that are not function stack local
but these are no problem if one works carefully

- Your work is very professional, no doubt. Mine is not. This is not my profession. It's my lifelong hobby. That's why I do it my way. I won't force anybody else to do it my way. My main objective is not to just complete the program, but to enjoy making it. If I can't enjoy making it, I don't do it. This is not meant to be rude, just synthetic and clear. I hope you understand.

our worlds are just too different - im currently working on 2 nearly
1 million lines of Code projects, multiplatform desktop, embedded, UIs, 3D, partialy very low level, multy system deplyment etc...
C/C++/Java/C#,Reversing,assembler etc. all of it and with around 10 3rd-party dependencies from Qt,boost,AWS,and,and,and - all stuff that you cannot
easily develop on your own in a decent time with a decent feature set, working in/guarding teams of 5-20 developers etc.
your pain with complexity and size of third parties or different languages are just far away from my work-livelong daily business - i just don't have
any pains at all - as long as no constraints like development speed, resource usage, quality, etc. are harmed

- "The way it works" is relative. It's the way some people do it. They may be many, but that doesn't mean the rest is wrong. When the first few people thought the Earth was round, most people were wrong. That happens all the time. Also, there's no need for one of the "two groups" to be wrong. They may both be right.

just two worlds - im living in a world wich is populated primarily with other developers of that background

- What do you mean when you say SDL was made because every attempt I would make at creating a portable library would fail?  Ha, ha. Anyway, what I want is so, so, so much smaller than SDL that I can easily do it. The only reason I haven't is I don't know how because I don't know the protocols and they are not very well documented. I can find out, but now I'm so busy!

everyone is able to do everything - thats not the question - the question is how long does it take, is it really needed, what does it really help the project
you can target what you want - its your hobby - but the restunts project is that big that any talking about the evilness of 3rd-parties is just sensless - you can start
with SDL, OpenGL, finish the project and then replace all the stuff with integer-only-x64-MIPS-AMIGA-inline assembler if you want to - because its not the primary code part
(if you write a wrapper around it - what a pro developer would always do)
you arguments are just only relevant to you - as i've never got any problems with 3rd-party and also never got problems in distribution of software
and i also find rarly programs that are graphical that don't use SDL or OpenGL, etc. so whats the point of beeing the only one concernd?
the answer: you are just not used to them (and their bytesize ), thats it, and your arguments in that area are just there to re-comfirm that

- I don't think SDL is bad. If it's non-OOP code in C, that sounds good to me.... as long as I can statically link it in my programs. But again, it's too big. It wouldn't make sense for small projects. There is one thing I don't know: whether I have to include the whole library source code with my program source code if I statically link it to it (to follow the GPL). Do you know that?

i don't know, but SDL2 comes with static libs and shared libs

Code Select Expand

Permission is granted to anyone to use this software for any purpose,
including commercial applications, and to alter it and redistribute it
freely, subject to the following restrictions:
1. The origin of this software must not be misrepresented; you must not
   claim that you wrote the original software. If you use this software
   in a product, an acknowledgment in the product documentation would be
   appreciated but is not required.
2. Altered source versions must be plainly marked as such, and must not be
   misrepresented as being the original software.
3. This notice may not be removed or altered from any source distribution.

so you can do what you want - even not mentioning the usage of it

- There are good reasons for choosing to work on the assembly code instead of the C code depending on what you want to do.
Mainly, that the C code is not complete yet and that, while the assembly is a very faithful representation of the original binary code,
the C code is an interpretation of it, albeit a very good one.
Sometimes, you will definitely want the C code, but many times, you will want the opposite for these reasons. It depends on what you want to do.

im talking about adding new C code to the asmorig-Code - not to base on the restunts-C code

- Portability is not a concern at this point, in my opinion, since this is real mode code.
Only if you get to the point to completely abstract the code from the memory structure,
will you be able to consider portability... unless you're thinking of a specialised emulator.

in assembler you see the algorithm AND memory model in code
in C you see only the algorithm (that would easily 5 times smaller and better to read)

for example the Watcom C compiler got a feature that allows to have all your variables in your own code-segment (im using that feature in my (still not finished) MT32.DRV port)
then you can write the C code, create a object from it and link that to the asmorig code without the need
to cope with not changing the binary layout to much or the need to add data to the original data segment
or trick your data into your functions

but maybe its just a knowledge problem here: you maybe know how to do that stuff in the pure asm, but not how to
do it with C

[Daniel3D]

- There are good reasons for choosing to work on the assembly code instead of the C code depending on what you want to do.
Mainly, that the C code is not complete yet and that, while the assembly is a very faithful representation of the original binary code,
the C code is an interpretation of it, albeit a very good one.
Sometimes, you will definitely want the C code, but many times, you will want the opposite for these reasons. It depends on what you want to do.

im talking about adding new C code to the asmorig-Code - not to base on the restunts-C code

I was hoping that that would be possible.

- Portability is not a concern at this point, in my opinion, since this is real mode code.
Only if you get to the point to completely abstract the code from the memory structure,
will you be able to consider portability... unless you're thinking of a specialised emulator.

in assembler you see the algorithm AND memory model in code
in C you see only the algorithm (that would easily 5 times smaller and better to read)

for example the Watcom C compiler got a feature that allows to have all your variables in your own code-segment (im using that feature in my (still not finished) MT32.DRV port)
then you can write the C code, create a object from it and link that to the asmorig code without the need
to cope with not changing the binary layout to much or the need to add data to the original data segment
or trick your data into your functions

but maybe its just a knowledge problem here: you maybe know how to do that stuff in the pure asm, but not how to
do it with C

It its always a knowledge problem. I don't think it is possible to know everything anymore. But in this case I believe you (llm :Red) have a wealth of knowledge that is much appreciated.

I hope somebody else besides me has time to dig soon. Finding a few isolated references to play with is one thing.
Identifying the whole main menu and all ...

Quoting CAS his estimate on the matter here
there must be a loop and in it, a function dispatcher. There must be a variable that holds the "current option" and a key selector to check and see if this value has to be incremented or decremented. Then, there will be some call to a dedicated function to read the status of the mouse buttons and coordinates. This could be a bare call to int 33h or an actual call to some internal routine that processes this information. After that, there will be comparisons to check whether the mouse pointer is inside any of the button areas. This will require a "for" loop that counts to five and in assembly, will look a little bit long in the main loop. This causes a change in the current option.

That is a whole different story, and I'm not sure if I can do that with my level of knowledge.

[Cas]

Looking up the instructions that actually do the thing is very complex and would take a lot of time. This is why I've been saying that the way to do this is to completely replace the routine. Even that is a lot of work, but is much more feasible.

And llm, yes, two different worlds. I have no "problem with" 3rd party software. I just have almost null interest in it. I play a game and I play it for fun and my game is code everything. If can do it, I do it. If it's not possible, I don't go for any alternatives, because it's not my game and I'm not interested. It's boring to me. What really is surprising is how people like often people push so hard to try to make me do things their way. I don't do the same to them. I wonder why that happens. They don't win anything with that.

I wanted to make my own tournament page from scratch without using libraries and even without databases. I did it and it works well. I wanted to make my own editor from scratch without a toolbox, just make my own. I did it. Now I'm making a car editing program my own way and it's in an early stage, but as you can see, it's improving. It looks like my way of doing the things does work

[Daniel3D]

And llm, yes, two different worlds. I have no "problem with" 3rd party software. I just have almost null interest in it. I play a game and I play it for fun and my game is code everything. If can do it, I do it. If it's not possible, I don't go for any alternatives, because it's not my game and I'm not interested. It's boring to me. What really is surprising is how people like often people push so hard to try to make me do things their way. I don't do the same to them. I wonder why that happens. They don't win anything with that.

I don't think they are trying to push you to do things their way. I believe they see you struggling and try to help, and they naturally suggest the way they know.
It's difficult to understand that for you, it is the journey and the struggle that makes it fun and worth doing.

And to be honest, I want to see the new game engine become a success. But doing everything yourself makes it also much harder to do so.

[Cas]

To me, there's no fun in putting pieces together. Say I'm a piece maker. Of course, there's the making a few pieces and putting them together with others, but the more putting pieces together and the less coding there is in a project, the less interesting it is to me and I just don't want to do it.

Before beginning with my new job, I've been getting more and more into electronics, home-made computers and micro-controllers. I think with time, I'll end up programming only those things. It's a lot of fun!