How To's For Developing II

[jakdillard]

First off I want to make a tutorial thread for potential and up-and-coming developers.
I do not intend for this thread to become a question pot for new members please if you are not familiar with rom creating or comfortable with the idea of royally messing up your phone please do not post here search first ...........

Fellow Developers if you have a tutorial that you want to share that is not a simple reg-edit please post it here .........

AGAIN THE IDEA FOR THIS THREAD IS TO BE A SHARING POT FOR DEVELOPERS PLEASE DO NOT POST HERE IF YOU ARE USING AUTOMATED TOOLS TO PORT OR WITH HOW TO CREATE A THEME

[jakdillard]

Porting SYS
Alright this first lesson is how to port a SYS folder from another device

The tools you will need for this is G'reloc your kitchen OEM folder and SYS folder and SYS folder from other rom.

Ok Step 1
Decompile your donor rom in your kitchen so that you have a SYS folder OEM folder and Rom Folder. Once you have this done place the G'reloc tool in the kitchen folder, run G'reloc , what this tool does is relocates the modules in the SYS folder using the info from the .vm imageinfo.bin, there are three important slots that you will need to get the info from they are slot 0: slot 1: and size of rom . What you will need to do now is write down those numbers exactly how they look for each slot.

Step 2

Move your Donor SYS folder out of your kitchen to a safe place (ex C:\desktop\Kitchen move donor to C:\SYS) then you want to get your new SYS folder from other rom (ex Kaiser SYS) and place it in your kitchen . Run G'reloc again and this time fill in the slots metioned above with the values of the last G'reloc run (ex slot 0: from 1st run value 01F30000 replace with new value slot 0 : 01FDE0000) then on G'reloc press the "doit!" button let it run until its done . Now on to building, add your oems in your donor OEM folder then build............(in some cases device may not boot because the new SYS will need the WINCE_NLS folder from donor SYS)

Troubleshooting

"Device will not go past HTC screen "
You must make sure the the WINCE_NLS folder is in you new SYS folder ........

[jakdillard]

I wrote this for a thread over at XDA-Developers, but no reason for them to have all the glory! (LennySh is translating a chinese tutorial that has pictures, if this isn't detailed enough for you)

How to Port a XIP

Your steps should look something like this :
NOTE: This does not cover more advanced topics. This is the BARE MINIMUM of steps to reliably get a working ported xip from a reasonably similar device.

If you don't know how to use a hex editor, don't know how to do hex arithmetic, or don't know how to put your phone in bootloader STOP NOW.

1) get your base rom (preferably an original and not someone's frankenrom)
2) get your donor rom (preferably an original and not someone's frankenrom)
3) cut the xip section out of each rom using a hex editor. XIP sections start with ECEC and end with a bunch of FFFFFs (there are two XIP sections, you usually want the 2nd one starting at x31000 or x32000 and ending at the the start of IMGFS)
4) use xipport (there are two versions, make sure you get the later version) to dump your base xip (if you end up missing imageinfo.txt files, you are using the wrong xipport version)
5) use xipport to "Make Maps" of your base xip
6) use xipport to "make pkgs" of your base xip
7) rename the base xip out dir to out.base
8 )use xipport to dump your donor xip
9) use xipport to "make maps" of your donor xip
10) use xipport to "make pkgs" of your donor xip
11) rename the donor xip out dir to out.donor
12) copy out.base to out (or dump your base xip and make pkgs again)

at this point, you should have out.base, out.donor, and out (a copy of out.base)

13) delete the MSXipKernel and MSXIPKernelLTK folders from files and modules in your out dir
14) copy the MSXipKernel and MSXIPKernelLTK folders from files and modules in your out.donor dir to the same locations in your out dir

You are done with out.base and out.donor. Everything from here on is done in out.

Make some space for your new, bigger xip by deleteing the following:
15) delete hd.dll and hd.dll.txt from modules/MSXIPKernel
16) delete osaxst0.dll and osaxst0.dll.txt from modules/MSXIPKernel
YOU MAY NEED MORE SPACE DEPENDING ON YOUR BASE and DONOR
(Moving WinCEnls from XIP to SYS is a good way to get a bunch of space)

Use M'reloc to relocate the V(irtual base) and D(ata) sections of all your donor modules donor modules are the ones in modules/MSXipKernel and moules/MSXipKernelLTK
17) Use the Map.txt file to identify your base address
18 ) Use M'reloc to move the modules so they are contiguous. (you hex calculator can help you here...adress of prior module+length of prior module = start of next module)
19) update the (module)/imageinfo.txt file v= and d= entries with the new values you used in M'reloc
20) repeat 18 -19) for each donor modue, being careful to remove any overlaps.

DO NOT MOVE nk.exe. If you really must move it, you have to use M'reloc-nk.
Personally, I recommend against moving any of the OEMXIPkernel modules, but to each his own.

20) use xipport to "Undo" (this undoes the "Make Pkgs")
If xipport fails, you botched your directory structure up. Delete OUT and start again from step 12)

21) use xipport to "Make Maps"
If xipport fails, you botched your m'reloc assignments. Start again from 17)

22) search map.txt for !!!!!!! in first two sections (data and virtual base)
if you find any, go back to 17) to resolve.

At this point, you should have no !!!!!! in the first 2 sections of map.txt

23) Use xipport to ReAlloc P
24) use xipport to "Make Maps"
25) search map.txt for !!!!!!
if you find any, you need advanced help.

You have now "ported" your donor xip (assuming it works!).

26) use xipport to "Build XIP out.bin"
27) use xipport to "Insert XIP" into your base rom at the XIP address you cut the base xip from in step 3)

28 )flash your rom.
If it boots, check settings/system/about version numbers for your donor xip build number.

If the build number does not match the donor rom, you screwed up 14), 26), or 27)
If it won't boot, you either screwed something up or your donor xip is not compatible. Try again or seek advanced help. Once you understand the technique, steps 21)-27) can be done while m'relocing to test your changes incrementally.

[jakdillard]

taken ......

[boggsie]

BePe's platform rebuilder and updated kitchen make SYS and XIP porting a breeze.

The platformrebuilder and kitchen thread:
http://darkforcesteam.com/showthread.php?t=569

Videos that he put together to:

Build your initial rom kitchen
http://www.bepe80.com/test/1

Port a new SYS & XIP into the stock rom kitchen
http://www.bepe80.com/test/2

Best regards,
-boggsie

PS - If this falls under the automated tools category then have mindfrost delete the post

[jakdillard]

in your xip the is a module called "nk.exe" open the "s000" file in the hexeditor of choice and then go to address 00003600 there you will see a line that has the ROM version which will be 1.0x "x" being the number that you have anything past the 1 is editable

[jakdillard]

Gain more Storage Memory (increase imgfs size)
There are 4 partitions in Diamond ROMs
part00 – ULDR
part01 – XIP
part02 – IMGFS
part03 – FAT (This partition exists only on few devices)

We all port XIP from different devices to exclude few modules to gain space and to upgrade the kernel and make the XIP partition smaller in size. Although the new XIP is smaller in size but because of the insertion addresses of XIP & imgfs, there is a gap of wasted space filled with FF between end of XIP & start of imgfs. Although there is no way we can include this space into XIP as free RAM but make use of this space in imgfs and gain whatever storage space we can

Files used as example for this tutorial
xip_out.bin: My own ported XIP of size (30CA12 in Hex, 3195154 in bytes)
os.nb.payload: My own cooked payload (since I also wanted the final ROM to be a cleaner ROM)
imgfs start: in my payload at 0x7A0000 (unedited)
XIP start: in my payload at 0x320000 (unedited)

Before we move into hex editing, let me give an overall outlook of the MBR & MSFLSH regions of the ROM
MBR is the Master Boot Record of the ROM (512 bytes) from 0x0 to 0x1FF. The infomation of partitions types Flags in hex offsets are called from the registry entry mentioned in boot.rgu below

The starting block (LBA) and number of sectors for each partition are defined as shown below

part00. 1C6 – 1C9 (starting block) 1CA – 1CD (number of sectors)
part01. 1D6 – 1D9 (starting block) 1DA – 1DD (number of sectors)
part02. 1E6 – 1E9 (starting block) 1EA – 1ED (number of sectors)
part03. 1F6 – 1F9 (starting block) 1FA – 1FD (number of sectors)

[HKEY_LOCAL_MACHINE\System\StorageManager\Partition Table]
"04"="FATFS" ; (hex: 1F2)
"20"="BOOT" ; (hex: 1C2)
"23"="RAWFS" ; (hex: 1D2)
"25"="IMGFS" ; (hex: 1E2)

MSFLSH50 is the Flash region of imgfs from 0x800 (see post #8 for screenshots, shown here is for Diamond) to 0xFFF. The starting block of imgfs is located in MSFLSH at 81C
e.g. if your device ROM's sector size is 200 then the MSFLSH50 region will starts at 0x200 and so on

Moving into the hex editing mode for making use of the wasted space between the actual XIP end & start of imgfs partitions

The new xip_out.bin is 30CA12 in total size (check your actual xip_out.bin size, shown here is just example) starting at 0x320000 (check you device XIP start, shown here is for Diamond) and ideally should end at 62CA12. But since the starting block of imgfs must be divisible by 20000 (see post #8 for screenshots, shown here is for Diamond) the imgfs needs to start at 640000. So the new XIP will have to be inserted into the payload at 0x320000 till 0x640000 with XIP size of 320000 and reduced wastage of 135EE bytes
The imgfs can also start at 630000 since this is directly after the XIP and also divisible by 20000, used here is 640000 as expansion for future xip_out.bin

Open the existing os.nb.payload in hex editor. Delete everything from 0x640000 till 0x79FFFF. This will move the imgfs from 0x7A0000 to 0x640000. Since we are now moving the imgfs partition next to new XIP, the number of sectors in new XIP and new LBA of imgfs needs to be edited to the revised value in the MBR region

To calculate the new starting block of imgfs partition we need the number of sectors in new XIP. To calculate that, use the following method
In Hex calc
Number of sectors = size of partition / sector size
e.g. (new XIP) 320000 (shown above) / 800 (see post #8 for screenshots, shown here is for Diamond) = 0640
since the coding is in little endian, we have to reverse these values to 40 06 00 00
Go to offset 0x1DA and change the values to 40 06 till 1DB and then 00 00

Now realloc the LBA of imgfs since we revised the number of sectors in XIP and to calculate that, use this method
In Hex calc
Logical Block Address (LBA) = Previous Partition LBA + Previous Partition number of sectors
e.g. (XIP LBA) 0640 + (XIP no of sectors) 0640 = 0C80
since the coding is in little endian, we have to reverse these values to 80 0C 00 00
Go to offset 0x1E6 and change the values to 80 0C till 1E7 and then 00 00

Logical Block Address (LBA) should be equal to (Previous Partition LBA + Previous Partition number of sectors * Sector Size)
e.g. (XIP LBA) 0640 + (XIP no of sectors) 0640 * 800 (see post #8 for screenshots, shown here is for Diamond) = 640000 (size of imgfs partition)

Similarly to imgfs calculate and change the LBA of FAT at 1F6 and 1F7 using the default imgfs no of sectors (use these since the cooking tools will change these as per actual size)

We have changed the LBA and number of sectors in MBR, but the OS needs to know the block address of imgfs in MSFLSH50 region
To calculate that, use this method
In Hex calc
MSFLSH50 Block Address = imgfs partition starting address / 20000 (see post #8 for screenshots, shown here is for Diamond)
e.g. (imgfs starting address) 640000 (shown above) / 20000 = 32
Go to offset 0x81C and change the value to 32

Save and close the os.nb.payload file in hex editor. Insert the new XIP into this file using this command
“insert.exe -i xip_out.bin -o OS.nb.payload -d 0x00320000 -s 0x00320000” (check your insert start address, shown here is for Diamond)

To calculate the size of XIP from MBR, use this method
In Hex calc
Size of XIP = Number of Sectors * Sector Size
e.g. (if the no of sectors in little endian) 0640 (shown above) * 800 (see post #8 for screenshots, shown here is for Diamond) = 320000 (sector size for diamonds)

This value shall be the "-s" while using insert.exe tool and to calculate the start of the XIP, use this method
In Hex calc
XIP Start = imgfs Start + "-s"