[REspawn]

Ok,
Now that SA is out for the PC we need to look at the format of the IMG files so we can add custom objects and such to the game.
The SA:PC version of the IMG files are the same as the Playstation 2 versions.

I have been thinking about this for a while and i think the below way is the best way of dealing with the new format.
But to implement it we need to agree on some standards so that the different SA based IMG tools and can all work in harmony without destroying the end users files.

I will start with a quick explanation of the IMG format for GTA3 and Vice City so people can see how it has evolved and then i will explain the SA format.

Grand Theft Auto 3 / Vice City IMG Format
There are 2 files, the first is the .img which holds the files. These files are stored in blocks of 2048 bytes.
Even if a file is 34 bytes it will occupy a full 2048 byte block.

The second file is the .dir file.
This is like a listing of each file in the .img file, it holds the name, offset and the number of blocks taken up by the file.
The name of the file as you can guess is the name of the file.
The offset is its position inside the .img file, the best way to explain this is to think of a shelf, one shelf one there is a texture file and on shelf two there is a model file.
The shelf number is like the offset, the position on the shelf where you can find what your looking for.
The size of the file is the number of blocks the file is currently occupying inside the .img file.

So when we wanted to add a file to Vice City or GTA3 all we had to do was put the file at the end of the .img and add a new entry to the .dir file.
So now that you know how GTA3 and Vice City works lets look at San Andreas...

San Andreas IMG Format
There is no longer 2 separate files, they have been merged into one singular file.
The dir listing of what is inside the .img is now at the start of the .img file.
The formats are still the same, files are kept in contiguous 2048 byte blocks, the dir listing still has an offset, size (num of blocks) and a name.
The only difference is that at the very start of the file is "VER2" and then a number which is the total number of files contained within the IMG.

You can start to see a problem already, now when we want to add a file we have to create an dir listing for it which means moving the blocks containing the files.
So this is where we need to work together and devise a solution, and after some thinking i think that this is the best way...

Proposed solution...
Blank dir entries are kept inside the IMG.
When you want to add a file the IMG tool checks for blank entire and if one is there it uses it.
Files being added to the archive can be added to the end.
If a blank entry is not available then one is created.


Data: The blocks containing the files
IMG Information: The version number and number of files in the IMG
DIR Information: The dir listings, the offset, name and num of blocks used



Step 1.
The first file directly after the dir listing is moved to the very end of the file and its corresponding dir entire has its offset updated.


Step 2.
This free space is then used to create the maximum amount of dir listing entires.
Each 2048 byte block will give us 64 blank entires.


This is best way i can see, using this method we create empty spaces that can be used at a later date and we don't have the overhead of moving all the files just so we can create some blank spaces.
The only real standard we need to agree on is what we name the empty spaces, we need a universal file name so all IMG tools created from now on can recognize the blank spaces.
Something like "-X27-X27-X27-X27-X27-X27"
There is never going to be a file called this so it should be perfectly safe.

Anywho im gonna start by trying out this method tomorrow, but for now im gonna go play with my brand spanking new copy of SA.
If any one else, not just the other coders out there has any thoughts or suggestions on this topic or how we should go about it then please post it.
This is only a suggestion on a way we can handle the file.

All the best,
-Dave

[RainingAcid]

Hmm...I don't really understand what you're saying, but it's possible to make an IMG for San Andreas, correct? If so, I want to help, just let me know how I can help.

[REspawn]

QUOTE (RainingAcid @ Jun 9 2005, 14:54)

Hmm...I don't really understand what you're saying, but it's possible to make an IMG for San Andreas, correct? If so, I want to help, just let me know how I can help.

SA already has an IMG, we know how to read it, we just need to decide on how we are going to write to it so that we don't end up with conflicts down the line.
Thats all.

[steve-m]

QUOTE (REspawn @ Jun 9 2005, 16:50)

The only real standard we need to agree on is what we name the empty spaces, we need a universal file name so all IMG tools created from now on can recognize the blank spaces. Something like "-X27-X27-X27-X27-X27-X27" There is never going to be a file called this so it should be perfectly safe.

Why do you want to name them? The safest way would be to scan through the file offsets and filter out the lowest. This is where the first file starts, and everything before (except for the header) can be used for the directory, no matter if it was filled with garbage or zeros. On reading, the program would skip the free space anyway. And this would work with every upcoming img tool, no matter if it complies with any standards or not.
Sorry, I got it completely wrong. The block offsets are relative to the end of the file list, right? Sure, we would need empty file names then, and a file counter higher than the actual ammount of files. You could try to fill the whole 32 byte of each entry with zeros, the game and any other img tools should get zero strings and a filelength of 0 then, so it wouldn't cause any trouble. And all img tools wouldn't display those entry entries to the user, but keep them "in mind".

[illspirit]

Not sure I like the idea of moving the first chunk of data to the end to make room for more dir. Might cause some lagging and/or errors while the engine is scanning up and down to find things it expects to be in order. How about if on first opening of the img, the tools force a rebuild to move all the data down a bit? This could be checked via CRC or something to see if the file has been modded yet.

That said, there should (in a perfect world) also be an info box in the status bar somewhere that says how many dir entries are available/total. Whenever it reaches 0/X, the tools could/should/would prompt to rebuild and add entries, maybe even giving an option on how many entries to add. Maybe this check and prompt could also be done on first run instead of a CRC?

[aad]

Im just hoping someone makes a good working IMG tool that can reconise the IMG file and the Dir file and then makes it editable like its now with vice city and gta3. Maybe im wrong and maybe this will not be possible but i really like if this can be made.

[steve-m]

Moving the first file to the end is just a temporary thing. The filename is still in the right place in the directory, so after a proper rebuild, everything is in order again. And rebuilding is recommended anyway after adding lots of files.

[Un3462]

steve? what you said first seems more right to me. block offsets are absolute, not relative at all (at least in script.img). i still don't get why everyone's making such a problem out of this, it's pretty obvious.

[ThaKilla]

As I see it.. as long as we make sure to move the first file in the IMG to the end and update the Dir part for it... and as long as all the IMG tools use this idea than there shouldn't be any problems. But hopefully we can find a way to add a new IMG altogether.

Edit: Also we would need to check to see if after the first file move if there is enough space to just add a new dir entry or if we need to move another file.


This post has been edited by ThaKilla on Thursday, Jun 9 2005, 19:25

[DexX]

im not sure i see the problem; if a new model/col/whatever is added, rebuild the dir in a temporary file, then append the contents of the IMG to that file after the dir and header info, delete the gta3.img, and set the temp file as the new IMG.

or am i missing something?

[ThaKilla]

Well with a 300+ MB file it would take 30mins-1hr depending on the speed, where when u just move one file and create a spacer, it will only take a min or two.

[steve-m]

QUOTE (CyQ @ Jun 9 2005, 21:00)

steve? what you said first seems more right to me. block offsets are absolute, not relative at all (at least in script.img). i still don't get why everyone's making such a problem out of this, it's pretty obvious.

Ok, I checked again: 522240 bytes before the first file starts. That are exactly 255 blocks, including the 8 byte header. According to the directory, the first file starts at 255, so you are totally right, file offset is absolute.

That means, what I first said applies:

QUOTE

Why do you want to name them? The safest way would be to scan through the file offsets and filter out the lowest. This is where the first file starts, and everything before (except for the header) can be used for the directory, no matter if it was filled with garbage or zeros. On reading, the program would skip the free space anyway. And this would work with every upcoming img tool, no matter if it complies with any standards or not.

No dummy files needed then.

Edit:

QUOTE (ThaKilla @ Jun 9 2005, 21:39)

when u just move one file and create a spacer, it will only take a min or two.

rather a sec or two

[ThaKilla]

Ok....

I have edited my GTASA-UE to handle the IMG this way... but i cant test it till i get the game tomarrow. If any of you would like to test it for me than here is the link.

This is BETA and is mainly a proof of concept.

GTASA Ultimate Editor

IF you find any problems let me know, because i used the PS2 Img file to make this.

Edit: To test this you should only have to add some B.S. fiel to the Gta3.IMG and it should move the 1st file to the end and add the B.S. file after it.

This post has been edited by ThaKilla on Thursday, Jun 9 2005, 21:21

[Kuiosikle]

Well, adding files works Killa, I haven't tried replacing any yet though...




This post has been edited by Kuiosikle on Friday, Jun 10 2005, 02:51

[ThaKilla]

I was just looking threw the "DATA" directory of GTASA.. and noticed a file called "gta_quick.dat".. here are some lines from it...

CODE

# # Load IDEs first, then the models and after that the IPLs # IMG DATA\PATHS\CARREC.IMG # # Object types # IDE DATA\MAPS\generic\vegepart.IDE

well that is a good way to make your own IMG entries. (I Think )

[DexX]

i succesfully replaced a file, but my "Add" and "Delete" and grayed out, and unusable..

[Kuiosikle]

QUOTE (ashdexx @ Jun 9 2005, 22:59)

i succesfully replaced a file, but my "Add" and "Delete" and grayed out, and unusable..

Hmm, that's strange, they are selectable for me..

Killa said he made a few fixes and will release a new version tommorow.

Anyways, did some fooling around;





Proof of concept?.....

This post has been edited by Kuiosikle on Friday, Jun 10 2005, 06:17

[JernejL]

QUOTE (illspirit @ Jun 9 2005, 19:51)

Not sure I like the idea of moving the first chunk of data to the end to make room for more dir. Might cause some lagging and/or errors while the engine is scanning up and down to find things it expects to be in order. How about if on first opening of the img, the tools force a rebuild to move all the data down a bit? This could be checked via CRC or something to see if the file has been modded yet. That said, there should (in a perfect world) also be an info box in the status bar somewhere that says how many dir entries are available/total. Whenever it reaches 0/X, the tools could/should/would prompt to rebuild and add entries, maybe even giving an option on how many entries to add. Maybe this check and prompt could also be done on first run instead of a CRC?

not really, gta-sa has to seek to each file offset separately aniway, so there is absolutely no lag, i will also use this block-moving-for-more-dir-room method
in my tools as well, it is the only clean way without rebuilding the whole img,
imagine this like windows' defrag program, file blocks are anywhere..

[Stretchnutter]

my iRipper utitlity rebuilds the whole 900mb IMG file in 120 seconds on my machine.

Its using the same code as the PS2 IMG file.. no changes..so dont expect anything special


(think of it as notepad, you click Save, it rebuilds the whole file with the changes youve made and you can make several changes before saving)


also please note that the VB source is there for the taking if you need to change something. (my site in sig)

This post has been edited by Stretchnutter on Friday, Jun 10 2005, 08:56

[ThaKilla]

Well I have updated my GTASA-UE. It should be able to edit the DATA files now.

Also I went threw and cleaned up a few things.

GTASA Ultimate Editor