Rxe Conversion Is Possible Now
#1
Posted 11 January 2005 - 12:38 PM
We have successfully built a program which will easily convert .exe programs to rxe!
The program is CALiPSO dRXE, and I will have it released very soon, I just need to clean it up a bit as it has been in a purely functional "experimental tool" stage for a while. For your amusement and as a demonstration I have uploaded a rxe version of Duobab's Boxworld (the original one).
There are only minor features of the rxe format that have not been addressed in the program, but they are all infrequent. We have been having issues with all/most Borland compiled programs as they make some writes to the code block which is physically flash and in rxe terms is a big critical issue. Boxworld contains a hack to workaround this problem but we have yet to make it a universal fix for all Borland programs, but it is a start. If you have information that may be useful in this regard do mention it.
Otherwise, I have successfully converted assembly programs and those from other compilers such as Digitial Mars.
Expect the first release of dRXE soon! Pick up Boxworld from file sharing!
- dscoshpe -
#2
Posted 11 January 2005 - 12:46 PM
So how much usable ram is left for an RXE program?
#3
Posted 11 January 2005 - 07:55 PM
Thank you so much :) That's a great breakthrough!
So how much usable ram is left for an RXE program?
I believe 260 bytes of RAM are used for the RXE code, so the rest of the open RAM is available for data. To get the most of RXE conversion, it might be beneficial to either store static data as code within assembly objects, or separate files loaded on demand - string constants and such will become part of the data segment and will be copied to RAM otherwise.
Also, we haven't tested with all the different borland C memory models yet... they may require different hacks than the ones boxworld needed, but again, digital mars doesn't seem to need any, and assembly would depend on the actual program (as long as no writes are done to the code segment, it should be fine).
#4
Posted 11 January 2005 - 09:49 PM
#5
Posted 12 January 2005 - 05:30 AM
dose this mean the Casio Elite are back in action?
I know dscoshpe's been working on this project, but I haven't done any casio stuff lately except helping debug the borland issues with dRXE. One of these days, I'll get around to fixing my link cable...
#6
Posted 12 January 2005 - 06:19 AM
I work on stuff off and on, like in streaks, and I happen to be in one now. The last time was probably in March. I suffer from Real-Life Syndrome and some of those days I just cant sit in front of a computer long enough to concentrate. But dRXE is a program I have wanted to finish for a while (I started it like 2-3 years ago) because it hasn't been done yet and so I have chipped away at it gradually. I plan on releasing one more version of AIP which will probably have RXE conversion built in because of the difficulties associated with converting programs seperately and then combining them on a drive, but I dont think that case will be in demand too much. Commander is, of course, the other never ending story (and a tragedy to me) but I don't know if it will ever see the light of day, but I am fond of the unique way it handles transfers, so who knows.
So I'm saying that you should check up with us from time to time. I know everytime someone asks I always open up the source and see where I was which then sometimes leads to getting something done!
PS - About Casio Elite the website, I think I'll try to dump it somewhere safe. It always seems to have bad luck with hosts. It will be a closed site though. I think the site I had with the assembly archive closed . (for those of you that remember it)
- dscoshpe -
#7
Posted 12 January 2005 - 02:08 PM
I hope we'll be able to use the conversions soon
Maybe crimson can host your page ...
#8
Posted 12 January 2005 - 04:01 PM
ex. I could use ~40 kb of RAM more in MLCafx since MLCafx (un-compressed) takes ~40 kb
I would be ~ 2 times RAM memory more then I can use in MLCafx now
I hope you'll finish it as soon as possible.
btw. I use tcc to compile the MLCafx but I could port it to DM if it's really needed
way to go dscoshpe!!!
#9
Posted 13 January 2005 - 07:38 AM
4nic8: As of right now it would be a good idea to port to Digital Mars because the hack hasn't been worked out as a universal solution. Boxworld works because we spent sometime analyzing it to determine what to do but we can't do that for every program. Besides, I think DM gives you a natural speed increase over Borland (Borland has lots of messy init code - which is also part of the problem for us).
But! Neither Brad or I have much expertise with the Borland compilers so maybe there a memory model or option that can be set to avoid the problem completely? I think maybe I will add a function to detect if a hack would be needed.
- dscoshpe -
#10
Posted 13 January 2005 - 09:32 AM
- dscoshpe -
#11
Posted 13 January 2005 - 10:57 AM
The RXE base address corresponds directly to the location in the flash (ie - drive letter) so to target an RXE for another drive, add 2000h per letter on top of the default L.. I always tested on drive L, and I tried something on O and it didnt run (after running on L) so I may have forgotten somehting in the conversion in this regard (or made something static )
A future version will make this more clear, and I will fix or resolve the issue with different drives.
- dscoshpe -
#12
Posted 13 January 2005 - 12:01 PM
#13
Posted 13 January 2005 - 06:12 PM
#14
Posted 13 January 2005 - 08:02 PM
Is it possible to change this timeout? I know you can change the timeout in the PHP config file, but can it be changed for a specific scripts?The 500 server error is because a PHP script cannot run more than 30 seconds... So it depends how fast you can send the file.
I noticed that classpad.net uses PHP and doesn't have this problem with some really big downloads. Ricky somehow worked around this problem on classpad.org and cpsdk.com. I'll find out how he did it.
#15
Posted 13 January 2005 - 08:49 PM
#16
Posted 14 January 2005 - 12:23 AM
- dscoshpe -
#17
Posted 14 January 2005 - 03:32 AM
Brad assured me that it is not neccessary to change the base because of how the flash is handled on the calc, so that simplifies that. The base never has to change except to specify a new place in the rom-disk image (if there are more than 1 files). The problem must be something limited to 4nic8's program. Boxworld and another program compiled with DM worked from other drives.
All is well now except for the Borland hack which I will be working on.
- dscoshpe -
#18
Posted 14 January 2005 - 07:33 AM
My program works very well! In fact I used only Orwell's gxlib and hooked animation engine to 0x1C. I haven't made anything special there, so I guess that problem will come back with some other game...The problem must be something limited to 4nic8's program.
#19
Posted 14 January 2005 - 12:14 PM
anyone tested them?!?!? maybe I'll do
#20
Posted 14 January 2005 - 04:02 PM
Now it works with all drives
So what was wrong then?
HUH surely it wasn't my fault . I changed dmc options from
-2 -Aa -Ab -msd -gg -o+spaceto
-2 -Aa -Ab -cpp -w7 -msd -gg -o+spaceand now my program is forced to be C++ (even it's written in C - ).
You can try it on your own (there is no warranty):
dRXE version (this must be at the top of a drive - but now it works with L:-Q: drives):
http://max.iem.pw.ed...xe/attt.exe.rxe
UPX-ED normal exe version:
http://max.iem.pw.ed...wj/drxe/ttt.exe
btw. strange thing:
C command coreleft (returns a measure of unused memory) doesn't seem to work fine with dRXE:
- in the emulator it gives 0 - but everything works fine
- on calc it gives the same value as with normal exe file
any cl00s why's that? / any way to test *real* left ram space?
#21
Posted 14 January 2005 - 04:38 PM
It gave me a Runtime Error 9 (Out of valid Range).
I can give you the data if this helps you.
#22
Posted 14 January 2005 - 04:46 PM
#23
Posted 14 January 2005 - 10:52 PM
For assembly, you *have to* manually declare the start of the code block in the file image because there is nothing to search for in the file that will hint to where it is (short of a full dissassembly). You can also do this for Pascal programs to get them to work. I've been thinking about adding .map file support to make this easier.
4nic8: Good job
- dscoshpe -
#24
Posted 15 January 2005 - 12:17 PM
make a function where the program can search a term like "RXE_START" that
the coder (in asm) would have to include at the beginning of the code block.
This could be usefull and used for easy codeblock identification ...
I'll e-mail you the test program.
#25
Posted 16 January 2005 - 01:14 PM
I forgot to mention that the sample you sent converted fine and ran on the calc.
- dscoshpe -
#26
Posted 16 January 2005 - 02:34 PM
this will make stuff a HUGE deal easier. You could try some of the programs
made by marco since he is coding is Pascal
#27
Posted 16 January 2005 - 04:15 PM
SEGMENT _TEXT class=CODE %include "LIBS\macros.asm" ;============================== RXE_START: start: mov ax,_TEXT mov ds,ax mov es,ax mov ax,stack mov ss,ax mov sp,stack_end call init_graphics_mode ............ ............
#28
Posted 16 January 2005 - 05:06 PM
it would look like the folowing
... db 'RXE_START' start: ;CODE .... end startthe stack could also be used like alway with the .STACK statement.
the same is valid for the .DATA & .CODE statements
#29
Posted 16 January 2005 - 11:19 PM
#30
Posted 16 January 2005 - 11:59 PM
As a reference: In RXE, only the data block is moved to ram when run by dos, the code block stays resident on the flash. Ideally you want a program with a code block which is as large as possible. The <{GNULINUX}>.exe example is very good in that regard, only a few hundred bytes makeup the data block! So virtually all of the program executes from flash!
Anyways, perhaps the RXE_START method can be used for that just as easily? (Sorry but my assembly knowledge doesn't answer that question for me)
- dscoshpe -
#31
Posted 17 January 2005 - 10:02 AM
.DATA db 'RXE_START' ;insert data here (include files with data also here and files with ;procedures in code segment) .CODE start: ;Insert code here end start
#32
Posted 18 January 2005 - 09:47 AM
It now has bug fixes, Pascal support, "DATA_START" support, and an experimental hack for Borland 2 (3 is coming soon....)
Pascal support: I couldn't figure out a way for having 100% success in figuring out where the data block should be short of a full disassembly analysis which Im not willing to do. So I compromised on assuming that every Pascal program's data block starts with 2 bytes 0000h which seems to be true, and that I have better odds of landing on the data block (since there are usually multiple instances of 0000h) if I figure the bytes to be positioned at x * 10h and to have 2 preceding bytes of 0h, in addition to this I take the first instance I find. I checked all of Marco's progs that I could and only one of them (sound.exe) did not succeed. I have dRXE list the other possibilities for your own reference, and for use in specifying manually.
"DATA_START": To implement this idea easier I changed it to "DATA_START" so it would be 10 bytes long. To use, just make sure it appears at the start of the data block and dRXE will mark its position as just that.
Borland 2 hack: Very experimental and relatively untested. Boxworld should work 100% with it but other than that I'm not sure because I just finished it before uploading for you. I was going to disable it again but I figured it might be fun. Remember, its only for Borland 2, if you forget, dRXE will remind you
You will also find you can now add and remove fix handler strings (the things in the "Fixes" tab) as well as edit them. Its really useful to know what their use is for forcing a program to convert properly without waiting for an update by me.
Also, I have made many more bug fixes which will help de-VB this VB program. (things like unsigning certain variables etc)
- dscoshpe -
#33
Posted 01 February 2005 - 08:42 AM
dRXE now supports fixing of stack pushs (which it hasn't so far) which should make the Borland hacks more reliable as we dont have to guess at which registers are safe to use.
dRXE will also now append a small chunk of information on the end of generated files for use in other programs. This will enable RXE's to be relocated after the conversion process is done making them able to run from within a drive rather than at the begining of it. It will require programs to be converted with the new version to take advantage of it.
On a similar note, Add-In Packager will get a new release as well...
New features include a more streamlined and more assuming interface. It also just so happens to support packaging of RXE's anywhere in a drive. If you are curious, there is no user intervention needed to work with RXE's in AIP as the work has been taken care of, meaning that the programs may remain in the form program.exe.rxe or whatever and be packaged as program.exe. The only thing that lets you know an RXE was packaged is a list of programs that were identified and adjusted as RXE's. Quite simple
There are many other less exciting changes to both programs too....
Expect these releases soon.
- dscoshpe -
#34
Posted 07 February 2005 - 10:31 AM
Borland 3 hack has been added with mixed results. Some programs convert wonderfully while others do not, I'm not sure what I'm missing but I will continue to investigate it.
A thing I call a 'dRXE Tag' is now placed on the end of each file converted and is meant to let programs that create rom-disk drives adjust the rxe files properly within the rom-disk. I have included a text file with the dRXE distribution called 'Relocation.txt' which explains how to use the dRXE tag. It's as easy to handle as possible I think.
A new Add-In Packager is being readied to support dRXE tags.
Only 1 more core conversion feature is left for me to include support for before dRXE is 100% compatible with the entire format!! In 1.53 I added support for 2 of these features. The last would be added by now but I have yet to encounter a situation that would need it.
- dscoshpe -
#35
Posted 12 February 2005 - 08:47 AM
I'll drop of notes if sth. doesn't work or goes wrong.
Btw. does RXE need the DOS interrupt to be working or can I disable it?
And could someone (glimpses at 2072) make sure that this infos are always posted on the french forum. The crteators of Flash100 linger there so it might me interresting for them to add this relocation thing to their program
Cu huhn
#36
Posted 12 February 2005 - 05:43 PM
#37
Posted 13 February 2005 - 11:47 AM
Aleph: This is not a form of compression. Essentially what an RXE tries to achieve is program that doesn't have to be completely loaded in to RAM from flash in order to run thus saving space that would have been used "needlessly". Dos compatible exes usually have 3 parts, the header, the code block (for the program instructions), and the data block (for game maps, graphics, etc). As an RXE the exe will loaded specially such that only the data block takes up RAM space and the code from the code block is executed directly from the flash, this can potentially free the entire amount of calc usable ram for other purposes, and if you need MORE space than that, you can start using special tricks so that your data that does NOT change during program execution becomes recognized as a part of the code block. If you want the nitty gritty, and my resource for understanding the idea see the RXE Theory of Operation straight from Datalight!
I'm going to put a file together that explains what really happens in more understandable terms and pictures and post it in the near future.
I enjoy these questions!
- dscoshpe -
#38
Posted 17 February 2005 - 08:56 AM
I want to know if the functions pointers like that:
void *(paf)(void);
would work as well?
(yes I'm in love with functions pointers )
#39
Posted 18 February 2005 - 01:25 PM
- dscoshpe -
#40
Posted 19 February 2005 - 08:52 AM
even perfectly that my program doesn't crash anymore in RXE
(instead of the EXE format it crashed a lot of time (memory alloc are strange... huhu in fact I forgot to free when the program ended!! yay thanks RXE, I can find my errors now )
0 user(s) are reading this topic
0 members, 0 guests, 0 anonymous users