How To Compile Your Own Kernels, Modules, Tinboot (NAND boot), and More for Beginners
 

This is a quick guide on getting started with compiling Android for the Touch Pro 2 from the git repositories. I am going to try to keep this short and simple, so I will be using my personal preference of software though others will work fine. This was made at the request of few people in the NAND testing thread, but I will try to include information for the non-NAND people too.

Required Tools
Virtualbox
Ubuntu (32bit is fine) Note: I recommend 10.04 for better stability with Virtualbox, but 10.10 does work just fine.
10 GB of disk space
High Speed Internet Connection
2 hours of time (about 30 minutes of actual interaction)

Installing Linux
First, install VirtualBox leaving everything as default. Now, click New and type Ubuntu 10.4 (or 10.10) for the name. Make sure Ubuntu is in the name so it auto-configures some options for us. For memory, I recommend 1 GB unless you have more than 4GB of RAM. For drives, leave the defaults and just keep clicking Next until it finishes.

Before you click Start, we want to optimize the settings and attach the ISO so it will boot from “CD”. To do this, Select Ubuntu 10.4 (or 10.10) and click Settings. Here are the settings you need to change:
Under the System section on the Motherboard tab, check Enable IO APIC, and on the Processor tab, Set Processor Count to the number of cores on your processor.

Under the Storage section, click on IDE Controller and change Type to ICH6. Now, click on the CD labeled Empty, then click the Folder with the green up arrow to open the Virtual Media Manager. Click Add and browse to the ISO you downloaded, click Open, click Select, and then click OK.
Now we can start the VM and install Ubuntu. Click Start and wait for the plum background with the keyboard at the bottom, press the down arrow and select Install Ubuntu. Keep clicking Forward until you can click Install Now. Follow the rest of the prompts while Ubuntu installs.

Preparing the Environment
Once Ubuntu is up and running, we need to install three software packages: Virtualbox Additions, git-core and wine. To install the VBox Additions, click Devices-CD/DVD Devices, and select VBoxGuestAdditions.iso. It never auto-runs for me, so I have to click Places, VBOXADDITIONS, then run autorun.sh, install and restart so we can copy/paste between Windows and Ubuntu :wink:

Terminal is where we spend the rest of our time, so click Applications-Accessories, and drag Terminal to the taskbar. Now open Terminal and copy/paste the following into it:

Note: If you prefer codesourcery's toolchain, you should download this one: Link

NAND users will also need the following:
Haret users will need the following:
Cloning these could take a while, especially for the prebuilt toolchain from Google. Just copy/paste and go make dinner :wink:

Compiling Kernel and Modules
Now that you have everything downloaded, we are ready to start compiling. The developers have done an awesome job with the Makefiles, so we only need to worry about the toolchain (to compile ARM code using an x86 processor) and the defconfig (configuration that determines what gets built).

If you wanted to simply compile the zImage and modules, using Terminal, copy/paste the following:
Your zImage will be located under ~/android/linux-msm-rhod-nand/arch/arm/boot/ and the modules will be output to ~/android/modules (or wherever specified by INSTALL_MOD_PATH).

Haret users should use the following:
On NAND, we are now putting all of the modules into a squash compressed file. This takes a little bit of setup, but it allows us to support multiple kernels. Keep in mind that this requires squashfs tools version 3.4. The modules need to be in the root of the squash file, and the name must be 2.6.27.46-01192-version.sqsh.

To set upsquashfs tools, download squashfs tools from here: ftp://ftp.slax.org/source/slax/sqlzm...shfs3.4.tar.gz
Now, extract the files, then browse into the squashfs3.4/squashfs-tools folder, then type make, then sudo make install.

On Haret, they load the modules from a tar.gz on in /sdcard/andboot. The name will be modules-2.6.27.46-01192-version.tar.gz. To get the version, try running:


androidupdate.tgz
Since the modules are now applied through androidupdate.tgz, you will need to make a directory structure in ~/android to look like this:

~/android
|--- androidupdate
|--- data
|--- system
|--- lib
|--- modules

The modules squash file should be put under /androidupdate/system/lib/modules. Also, any files that you want updated in either data or system can be placed in here. For example, if you want to include certain APK's, you can put them under /system/app or /data/app.

Compiling Initrd.gz
For NAND, we have our own initrd.gz in the bootenv git that we cloned. We try to rebuild these and commit them every time we update bootenv, but sometimes the change doesn't affect enough rebuild and upload. So, I always recommend rebuilding initrd.gz fresh and copying that into ~/android/tinboot-linux-msm/kernel. You can build this very easily by simply browsing to ~/android/bootenv and running sh buildInitrd.sh

Compiling Tinboot
Tinboot has been changed significantly to only use 1 cmdline for all devices :). If you are compiling for NAND, you will need to use the initrd.gz from the bootenv git we cloned. If you are compiling for Haret, you will need the initrd.gz from Main Page - The XDAndroid Project. Just grab the latest and copy that along with the zImage into ~/android/tinboot-linux-msm/kernel. If you want to overclock the processor, change the lcd density, or change anything else, you can modify ~/android/tinboot-linux-msm/tinboot/tinboot2.S

Changing this file is extremely simple, but you can really mess things up if you change some of the code. I would only recommend changing the line that starts with .asciz. So, let's open tinboot2.S in gedit, find .asciz, and make the changes to the command line. If you want to overclock, add acpuclock.oc_freq_khz=716800 (change number to whatever overclock your phone can handle). If you are cooking a ROM, you can change the mtdparts to change the size of the partitions. All values are in HEX, and I will have a separate post for NAND chefs that will explain this much better.

We are now ready to compile. Unfortunately, the scripts on git require quite a bit of tweaking to get working. For now, try using the full build or tinboot script that I have posted in the second post.

Making the RootFS (Haret Only)
The RootFS is a little different since there is no simple make command. The current way to "compile" the RootFS is through the script located here: generate-rootfs.sh

Attempting to modify this script always ended up giving me weird genext2fs warnings, and I found it to be much cleaner to just use the script as is. Instead, I choose to inject my modified init file right before genext2fs creates rootfs.img. Also, note that the script will throw a warning about .git not found or something similar. That is something from the gitclean.sh and can be ignored.

The only catch here is that you have to keep an eye out for updates to init as you will be overwriting any future updates. I used to keep a simple text file with my script that keeps track of what changes I made to init so I could easily add them back in the future if init was updated.

Re: How To Compile Your Own Kernels, Modules, Tinboot (for NAND booting), and More
 

Below is a copy of my full build script for NAND. Haret users will have to make a few changes which I'll explain later. Just copy/paste it into gedit and save as fullbuild.sh and run it from terminal. You may need to change the directory paths at the beginning if you don't put them where I recommended.



What the heck is zImage, modules, tinboot, rootfs, initrd, and System.ext2?
This section will be updated slowly as I have time.

The zImage is the kernel (which is required to run the operating system). We currently use the linux kernel 2.6.27.46.

The modules are your drivers. These are a matched pair with the zImage/kernel. It is also important to clean up the modules periodically when building the kernel/modules or you will start running out of space :)

Awesome explanation of the kernel and its modules here: http://www.howtogeek.com/howto/31632...at-does-it-do/

Tinboot is the bootloader for running Android from the NAND. Currently, we put initrd.gz and the zImage on the NAND while the zImage, rootfs, and System.ext2 are pulled from the SD Card.

RootFS is a special type of ramfs file system. It basically lets us use part of the memory as a partition for caching files. This gives us a nice performance boost :)

Initrd is an initial RAM disk that is temporary and used only for booting. Generally it uses a slightly different scheme than rootfs does.

System.ext2 is your file system. This is technically your hard drive where all of the Android files and applications are stored. It uses the ext2 file system which is an extended file system designed specifically for linux.

Re: How To Compile Your Own Kernels, Modules, Tinboot (NAND boot), and More for Begin
 

Thanks for the info on this Nate! :occasion5:

Re: How To Compile Your Own Kernels, Modules, Tinboot (NAND boot), and More for Begin
 

I've compiled for HARET before once. This really helps though.

Thanks!

Re: How To Compile Your Own Kernels, Modules, Tinboot (NAND boot), and More for Begin
 

Can't seem to get past this step:
What I see:
Everything else went good. Any Ideas?

Thanks!

Re: How To Compile Your Own Kernels, Modules, Tinboot (NAND boot), and More for Begin
 

Nice catch. It should have been htc_msm_nand_defconfig. Updating now :)

Re: How To Compile Your Own Kernels, Modules, Tinboot (NAND boot), and More for Begin
 

That seemed to work.
But the Next Step:
I get this:

Re: How To Compile Your Own Kernels, Modules, Tinboot (NAND boot), and More for Begin
 

Try make clean, then do the defconfig one more time, then the last make. Also, double check and make sure the arm-eabi-gcc is there. If not, do a git pull in the prebuilt directory.

Re: How To Compile Your Own Kernels, Modules, Tinboot (NAND boot), and More for Begin
 

FYI this "how to" is ACL approved :-)

:thumbright:

Re: How To Compile Your Own Kernels, Modules, Tinboot (NAND boot), and More for Begin
 

This is a GREAT thread, thanks Nate

Re: How To Compile Your Own Kernels, Modules, Tinboot (NAND boot), and More for Begin
 

Thanks for the feedback guys! Really glad I could help :)

Most of this is derived from the scripts in tinboot which I modified, cleaned up, and compiled into one script. I'm hoping to have a chance to finish the tinboot section and start on the rootfs section later tonight.

Re: How To Compile Your Own Kernels, Modules, Tinboot (NAND boot), and More for Begin
 

Ok did that but It still didn't work. :(
The prebuilt directory was up to date and the file is there too...

Is this working for everyone else?

Maybe it's in this step:
Here is my output:

Re: How To Compile Your Own Kernels, Modules, Tinboot (NAND boot), and More for Begin
 

That looks perfect. I just reverted to my original snapshot, so let me take another look at it from before I've even done the defconfig.

OK, I just ran each command exactly in order (I made the guide as I built it too), and everything worked by copy/pasting from the first post. There are two things that I can think of. First, there was an extra space between ARCH=arm and CROSS_COMPILE in the first post (which I just fixed), but I didn't remove it when I just now compiled and everything worked just fine. The second thought is, you didn't run Terminal as root or do sudo on the git clone did you? You could try typing !! and hit enter after it fails and see if it works elevated. If that's not it, I would go into android, type sudo rm -rf prebuilt, then close Terminal, open a new Terminal making sure you are not root, and clone prebuilt again :(

Re: How To Compile Your Own Kernels, Modules, Tinboot (NAND boot), and More for Begin
 

Ok! I had my Android Folder on my desktop and not in the home folder...:spin:

This allowed the make command not to fail right away...

Update:
It WORKS! User Error... Must have been something with ROOT. Sorry to bother!:toothy10:

Re: How To Compile Your Own Kernels, Modules, Tinboot (NAND boot), and More for Begin
 

Ahh, glad you were able to get it going. I try not to use root whenever possible :)

Was gonna say that if you put it on your desktop, you would need to change the CROSS_COMPILE path too :p

Re: How To Compile Your Own Kernels, Modules, Tinboot (NAND boot), and More for Begin
 

Because I'm a complete and total noob, I'm going to ask what certian things are just because I like seeping through code and learning a little. I've been outta school forever and never really did anythign with OS development, so this is all way over my head, but I'm going to try to understand it all. =P

First question in a series of quests would have to be what is an XIP. I read something about it being execute in place for a kernel. So it doesn't even have to load it into RAM or decompress. Is this the same thing? I know tinboot is a bootloader and what I read about XIP was for linux kernel.


Another question would have to be why is it that we are importing a blank image into the OS.nb file? Couldn't we put the rootfs.img into there instead? That way, it would be on the NAND and not the SD card. IDK, I'm kinda noob at this. I'm sure there's a reason, im just asking for all the different meathods to the madness. =P Just trying to learn. xD

Also, if it turns out that we are able to put the rootfs.img into the OS.nb, could we also put the modules in there? Again, I've never built ROM's for phones before, so I don't understand how this all works. =P I'm just looking for a way to possibally make SD card hot swappable.

Re: How To Compile Your Own Kernels, Modules, Tinboot (NAND boot), and More for Begin
 

As far as I know, the reason that stuff still has to be on the SD card is that they still haven't figured out how to get the internal storage to be recognized within Android. It prevents putting much of anything onto the internal flash storage and forces everything to still be on the SD card pretty much.

Re: How To Compile Your Own Kernels, Modules, Tinboot (NAND boot), and More for Begin
 

SUCCESS!

I used your build script on the second post. I did have to modify this line:

DIRECTORY=$(cd `dirname $0` && pwd)
To:
DIRECTORY=$ANDROID/tinboot-linux-msm/

Thanks for all your help on this!

Now lets see what i can do... :evil4:

Re: How To Compile Your Own Kernels, Modules, Tinboot (NAND boot), and More for Begin
 

Ahhh, that makes sense. Glad you were able to figure it out. I need to fix the script so it doesn't even use that :)

Edit: Fixed it up so it just uses the variable TINBOOT instead of DIRECTORY. Makes it better for those who launch scripts from a different directory than the script :)

Re: How To Compile Your Own Kernels, Modules, Tinboot (NAND boot), and More for Begin
 

HAHA Last time I built a kernel I downloaded the entire Android source from google then the haret git..... OMG did it take a while. :D

Re: How To Compile Your Own Kernels, Modules, Tinboot (NAND boot), and More for Begin
 

OK Ive tried everything I can think of. It begins to compile and at the last minuet I get this:

Outputting files to /home/edouble/android/output/sdcard
mv: cannot move `modules-2.6.27.46-01203-g4e5852fb.tar.gz' to `/home/edouble/android/output/sdcard': No such file or directory
Start compiling tinboot/xip for rhod210.....
/home/edouble/android/tinboot-linux-msm/tinboot/tinboot2.S: Assembler messages:
/home/edouble/android/tinboot-linux-msm/tinboot/tinboot2.S:115: Error: file not found: kernel/zImage
/home/edouble/android/tinboot-linux-msm/tinboot/tinboot2.S:117: Error: file not found: kernel/initrd.gz
/home/edouble/android/prebuilt/linux-x86/toolchain/arm-eabi-4.4.0/bin/arm-eabi-objcopy: 'tinboot.o': No such file
mv: cannot stat `tinbootxip': No such file or directory
rm: cannot remove `tinboot.o': No such file or directory
Inserting tinboot into payload
fullbuild.sh: 122: wine: not found
mv: cannot stat `os.nb.payload.NEW': No such file or directory
Inserting blank imgfs into payload
fullbuild.sh: 125: wine: not found
Creating os.nb portion of nbh
fullbuild.sh: 127: cannot open OS-new.nb.payload: No such file
Creating NBH
fullbuild.sh: 129: wine: not found
rm: cannot remove `OS-new.nb.payload': No such file or directory
rm: cannot remove `OS-new.nb': No such file or directory
mv: cannot stat `/home/edouble/android/tinboot-linux-msm/ruu_signed.NBH': No such file or directory
zip warning: name not matched: output

zip error: Nothing to do! (try: zip -r 20101029-2.6.27.46-01203-g4e5852fb.zip . -i output)
cd: 146: can't cd to android/tinboot-linux-msm/tinboot
rm: cannot remove `*.NBH': No such file or directory
cd: 148: can't cd to sdcard
rm: cannot remove `*.tar.gz': No such file or directory
edouble@ubuntu:~/android/tinboot-linux-msm/tinboot$


What am I missing. I followed the instructions to the letter. Im thinking this has something to do with where the compiler puts the .NHB file but Im not sure. Plz help.... Thanks....:cwm10:

Re: How To Compile Your Own Kernels, Modules, Tinboot (NAND boot), and More for Begin
 

So, two things. First, you still need to install wine. Just do a sudo apt-get install wine. Second, I need to fix a tiny bit in the script. I changed it up for people who didn't put the android folder in their home directory and forgot to fix the relative path on tinboot.

Edit: OK, pretty sure I found the issue with the XIP part. Once you get wine installed, I think you'll be all set :)

Re: How To Compile Your Own Kernels, Modules, Tinboot (NAND boot), and More for Begin
 

one thing not in the instructions that is needed is a:

mkdir /home/edouble/android/output/sdcard

also as natemcnutty said, you still need to install wine =D

Re: How To Compile Your Own Kernels, Modules, Tinboot (NAND boot), and More for Begin
 

There are a couple of folders that need to be made. One pointed above, and the XIP folder. You already created it but it was made in the android folder and not the tinboot-linux-msm like it should be.

Then the code will work great! Thanks again!

Re: How To Compile Your Own Kernels, Modules, Tinboot (NAND boot), and More for Begin
 

OK, installed wine and added the extra folders. I did start over to make sure everything was done correctly. I get to the end as before but this time I get:

Outputting files to /home/edouble/android/output/sdcard
Start compiling tinboot/xip for rhod210.....
mv: cannot move `tinbootxip' to `xip/rhod210': No such file or directory
Inserting tinboot into payload
mv: cannot stat `os.nb.payload.NEW': No such file or directory
Inserting blank imgfs into payload
Creating os.nb portion of nbh
Creating NBH
updating: output/ (stored 0%)
updating: output/sdcard/ (stored 0%)
updating: output/sdcard/modules-2.6.27.46-01204-ga0bebc2.tar.gz (deflated 1%)
updating: output/ruu_signed.NBH (deflated 11%)

The compiler stops there and then gives me the command prompt. I really feel like a noob but I really want to try this out. What am I missing?

Re: How To Compile Your Own Kernels, Modules, Tinboot (NAND boot), and More for Begin
 

I will try running the script with rhod210 when I get home if this isn't resolved. I myself ran it as rhod400, so I will see what different results I get when I run it as your model.

Edit:

Try finding this part of the code in the script:

and changing it to
that should clear up the first error, unless it cannot find tinbootxip. if thats the case, i will have to look at it more when i get home

as far as the
i will have to wait until i get home to look at. that'll be like 3 hours or so.

Re: How To Compile Your Own Kernels, Modules, Tinboot (NAND boot), and More for Begin
 

ahh man .. hope you dont brick yourself . i corrupted my phone already once and wozzer ended up needing a new phone. Just gotta be careful.

Re: How To Compile Your Own Kernels, Modules, Tinboot (NAND boot), and More for Begin
 

Add the lines of code to the script from my post above this and it should work out just fine.

Re: How To Compile Your Own Kernels, Modules, Tinboot (NAND boot), and More for Begin
 

Thanks. Yeah, all of the problems are because I removed the relative references. I always run my script from the tinboot directory, and my script was grabbing the pwd. I had this:

if [ ! -d "$TINBOOT/xip" ] ; then
mkdir xip
fi

instead of

if [ ! -d "$TINBOOT/xip" ] ; then
mkdir $TINBOOT/xip
fi

That should be fixed now. Also, I missed the fact that the buildxip.sh script in tinboot actually has if -a instead of if -d which will fail the build as well :(

Re: How To Compile Your Own Kernels, Modules, Tinboot (NAND boot), and More for Begin
 

OK, if you could, please try creating a new fullbuild.sh based on the updates I just made. I made sure that all pathing is correct regardless of where you launch the script from, and as long as you specify the locations of your sources in the beginning, it will work perfectly.

I tested moving my sources around, tested the new script, and compiling was still successful :)

Re: How To Compile Your Own Kernels, Modules, Tinboot (NAND boot), and More for Begin
 

Worked like a charm! Great job! :thumbleft:

Re: How To Compile Your Own Kernels, Modules, Tinboot (NAND boot), and More for Begin
 

Awesome. Now that we've got this working, I'll start going through some of the significant files that you might want to modify. Early tomorrow I'll try to add more to the second post about the tinboot2.S file and init for rootfs. There's a few simple modifications you can do to change things for your liking, and hopefully I'll learn a few new things in the process :)

Re: How To Compile Your Own Kernels, Modules, Tinboot (NAND boot), and More for Begin
 

I have to let it go at this point. Followed the instructions to the letter. I have gone through the compiler multiple times and I get nothing. Just for FYI this is the final output from the compiler:


Thing is the referring files are empty. I know im doing something wrong so Im going to let it go. I will try the prebuilt NHB play with that. Keep up the work guys....

Re: How To Compile Your Own Kernels, Modules, Tinboot (NAND boot), and More for Begin
 

Check in your Android folder for a zip file. Or if you made a Dropbox folder check in there.
Everything looks fine from what you have showed.

Good luck!

Re: How To Compile Your Own Kernels, Modules, Tinboot (NAND boot), and More for Begin
 

Yeah, I'll put an echo at the end of the script to say where it is moving the files to. I love getting a chance to make this a little more user friendly :)

Edit: Script has been updated :) Also fixed an issue where the sdcard folder may not have been created if you already had the output folder but not the sdcard folder.

Re: How To Compile Your Own Kernels, Modules, Tinboot (NAND boot), and More for Begin
 

what happens if you replace the blank filesystem with the rootfs.img? im going to experiment with this =P i guess there would be code in the kernel to not look for that file then... hmm...we shall play around and see what happens...

edit:
haha, thats a stupid idea... the initrd will still look for the rootfs on the SD card... hmm... i wonder if tinboot needs to be modded in any way for it to put the rootfs on the nand...

Re: How To Compile Your Own Kernels, Modules, Tinboot (NAND boot), and More for Begin
 

I would assume you would need to merge the rootfs into the nbh file with tinboot. How it would be addressed I am unsure. Then you need to worry about the system.ext2 and data.img. I am sure someone that does custom android roms for other phones can give an idea how to cook this.

Re: How To Compile Your Own Kernels, Modules, Tinboot (NAND boot), and More for Begin
 

right, but on the other phones, the bootloader that is already on the phone allocates all that already. that is something they don't have to worry about. and if i'm not mistaken, rootfs.img would mount to the "/" directory and the data.img would mount to the "/data/" directory. idk where system.ext2 would mount...

but anyway, it might have to be different partitions on the nand. I think that when the NHB gets written to the nand, the .img/.bin file gets written as a pseudo-partition. just have to get the phone to know its there xD

Re: How To Compile Your Own Kernels, Modules, Tinboot (NAND boot), and More for Begin
 

Yeah, it's more of the changes to initrd and other components that rely on rootfs that I'm curious about. I know how to use yang to add it to the NBH, it's actually referencing it that I don't know how to do :)

Re: How To Compile Your Own Kernels, Modules, Tinboot (NAND boot), and More for Begin
 

How did the Vogue Devs do it? It seems that they are the reference for any other device because it was the first. =P