PPCGeeks - How To Compile Your Own Kernels, Modules, Tinboot (NAND boot), and More for Beginners

PPCGeeks (http://forum.ppcgeeks.com/index.php)

- Android On TP2 (http://forum.ppcgeeks.com/forumdisplay.php?f=179)

How To Compile Your Own Kernels, Modules, Tinboot (NAND boot), and More for Beginners

This is a quick guide on getting started with compiling Android for the Touch Pro 2 from the git repositories. I am going to try to keep this short and simple, so I will be using my personal preference of software though others will work fine. This was made at the request of few people in the NAND testing thread, but I will try to include information for the non-NAND people too.

Required Tools
Virtualbox
Ubuntu (32bit is fine) Note: I recommend 10.04 for better stability with Virtualbox, but 10.10 does work just fine.
10 GB of disk space
High Speed Internet Connection
2 hours of time (about 30 minutes of actual interaction)

Installing Linux
First, install VirtualBox leaving everything as default. Now, click New and type Ubuntu 10.4 (or 10.10) for the name. Make sure Ubuntu is in the name so it auto-configures some options for us. For memory, I recommend 1 GB unless you have more than 4GB of RAM. For drives, leave the defaults and just keep clicking Next until it finishes.

Before you click Start, we want to optimize the settings and attach the ISO so it will boot from “CD”. To do this, Select Ubuntu 10.4 (or 10.10) and click Settings. Here are the settings you need to change:
Under the System section on the Motherboard tab, check Enable IO APIC, and on the Processor tab, Set Processor Count to the number of cores on your processor.

Under the Storage section, click on IDE Controller and change Type to ICH6. Now, click on the CD labeled Empty, then click the Folder with the green up arrow to open the Virtual Media Manager. Click Add and browse to the ISO you downloaded, click Open, click Select, and then click OK.
Now we can start the VM and install Ubuntu. Click Start and wait for the plum background with the keyboard at the bottom, press the down arrow and select Install Ubuntu. Keep clicking Forward until you can click Install Now. Follow the rest of the prompts while Ubuntu installs.

Preparing the Environment
Once Ubuntu is up and running, we need to install three software packages: Virtualbox Additions, git-core and wine. To install the VBox Additions, click Devices-CD/DVD Devices, and select VBoxGuestAdditions.iso. It never auto-runs for me, so I have to click Places, VBOXADDITIONS, then run autorun.sh, install and restart so we can copy/paste between Windows and Ubuntu :wink:

Terminal is where we spend the rest of our time, so click Applications-Accessories, and drag Terminal to the taskbar. Now open Terminal and copy/paste the following into it:

Code:

sudo apt-get install git-core

sudo apt-get install wine

mkdir android

cd android

git clone git://android.git.kernel.org/platform/prebuilt.git

Note: If you prefer codesourcery's toolchain, you should download this one: Link

NAND users will also need the following:

Code:

git clone git://gitorious.org/~acl/linux-on-qualcomm-s-msm/linux-msm-rhod-nand.git

git clone git://gitorious.org/tinboot-for-linux-msm/tinboot-linux-msm.git

git clone git://gitorious.org/tinboot-for-linux-msm/bootenv.git

Haret users will need the following:

Code:

git clone git://gitorious.org/linux-on-qualcomm-s-msm/linux-msm.git

git clone git://gitorious.org/xdandroid/rootfs.git

Cloning these could take a while, especially for the prebuilt toolchain from Google. Just copy/paste and go make dinner :wink:

Compiling Kernel and Modules
Now that you have everything downloaded, we are ready to start compiling. The developers have done an awesome job with the Makefiles, so we only need to worry about the toolchain (to compile ARM code using an x86 processor) and the defconfig (configuration that determines what gets built).

If you wanted to simply compile the zImage and modules, using Terminal, copy/paste the following:

Code:

cd android

mkdir modules

cd linux-msm-rhod-nand

make clean

make ARCH=arm htc_msm_nand_defconfig

make ARCH=arm CROSS_COMPILE=~/android/prebuilt/linux-x86/toolchain/arm-eabi-4.4.0/bin/arm-eabi- INSTALL_MOD_PATH=~/android/modules zImage modules modules_install

Your zImage will be located under ~/android/linux-msm-rhod-nand/arch/arm/boot/ and the modules will be output to ~/android/modules (or wherever specified by INSTALL_MOD_PATH).

Haret users should use the following:

Code:

cd android

mkdir modules

cd linux-msm

make clean

make ARCH=arm htc_msm_android_defconfig

make ARCH=arm CROSS_COMPILE=~/android/prebuilt/linux-x86/toolchain/arm-eabi-4.4.0/bin/arm-eabi-  INSTALL_MOD_PATH=~/android/modules zImage modules modules_install

On NAND, we are now putting all of the modules into a squash compressed file. This takes a little bit of setup, but it allows us to support multiple kernels. Keep in mind that this requires squashfs tools version 3.4. The modules need to be in the root of the squash file, and the name must be 2.6.27.46-01192-version.sqsh.

To set upsquashfs tools, download squashfs tools from here: ftp://ftp.slax.org/source/slax/sqlzm...shfs3.4.tar.gz
Now, extract the files, then browse into the squashfs3.4/squashfs-tools folder, then type make, then sudo make install.

On Haret, they load the modules from a tar.gz on in /sdcard/andboot. The name will be modules-2.6.27.46-01192-version.tar.gz. To get the version, try running:

Code:

cat ~/android/linux-msm-rhod-nand/include/config/kernel.release

androidupdate.tgz
Since the modules are now applied through androidupdate.tgz, you will need to make a directory structure in ~/android to look like this:

~/android
|--- androidupdate
|--- data
|--- system
|--- lib
|--- modules

The modules squash file should be put under /androidupdate/system/lib/modules. Also, any files that you want updated in either data or system can be placed in here. For example, if you want to include certain APK's, you can put them under /system/app or /data/app.

Compiling Initrd.gz
For NAND, we have our own initrd.gz in the bootenv git that we cloned. We try to rebuild these and commit them every time we update bootenv, but sometimes the change doesn't affect enough rebuild and upload. So, I always recommend rebuilding initrd.gz fresh and copying that into ~/android/tinboot-linux-msm/kernel. You can build this very easily by simply browsing to ~/android/bootenv and running sh buildInitrd.sh

Compiling Tinboot
Tinboot has been changed significantly to only use 1 cmdline for all devices :). If you are compiling for NAND, you will need to use the initrd.gz from the bootenv git we cloned. If you are compiling for Haret, you will need the initrd.gz from Main Page - The XDAndroid Project. Just grab the latest and copy that along with the zImage into ~/android/tinboot-linux-msm/kernel. If you want to overclock the processor, change the lcd density, or change anything else, you can modify ~/android/tinboot-linux-msm/tinboot/tinboot2.S

Changing this file is extremely simple, but you can really mess things up if you change some of the code. I would only recommend changing the line that starts with .asciz. So, let's open tinboot2.S in gedit, find .asciz, and make the changes to the command line. If you want to overclock, add acpuclock.oc_freq_khz=716800 (change number to whatever overclock your phone can handle). If you are cooking a ROM, you can change the mtdparts to change the size of the partitions. All values are in HEX, and I will have a separate post for NAND chefs that will explain this much better.

We are now ready to compile. Unfortunately, the scripts on git require quite a bit of tweaking to get working. For now, try using the full build or tinboot script that I have posted in the second post.

Making the RootFS (Haret Only)
The RootFS is a little different since there is no simple make command. The current way to "compile" the RootFS is through the script located here: generate-rootfs.sh

Attempting to modify this script always ended up giving me weird genext2fs warnings, and I found it to be much cleaner to just use the script as is. Instead, I choose to inject my modified init file right before genext2fs creates rootfs.img. Also, note that the script will throw a warning about .git not found or something similar. That is something from the gitclean.sh and can be ignored.

Code:

cd "${TMPDIR}"/"${REPODIR}"

./scripts/gitclean.sh

cp -a . "${TMPDIR}"/rootfs



# copy modified init file

rm "${TMPDIR}"/rootfs/init

cp "${ANDROID}"/rootfs/init "${TMPDIR}"/rootfs



chmod u+s "${TMPDIR}"/rootfs/bin/pppd "${TMPDIR}"/rootfs

The only catch here is that you have to keep an eye out for updates to init as you will be overwriting any future updates. I used to keep a simple text file with my script that keeps track of what changes I made to init so I could easily add them back in the future if init was updated.

Re: How To Compile Your Own Kernels, Modules, Tinboot (for NAND booting), and More

Below is a copy of my full build script for NAND. Haret users will have to make a few changes which I'll explain later. Just copy/paste it into gedit and save as fullbuild.sh and run it from terminal. You may need to change the directory paths at the beginning if you don't put them where I recommended.

Code:

#!/bin/sh

#-------------------------------------------------------------------------

# Build Script for linux-msm-rhod-nand (Android on HTC) kernel, modules, XIP, and NBH

#-------------------------------------------------------------------------

#

# Set the following variables

# ANDROID -- Root director for Android files

ANDROID=~/android

# TINBOOT -- tinboot directory

TINBOOT=$ANDROID/tinboot-linux-msm

# KERNEL_PATH -- directory containing the linux-msm kernel source

KERNEL_PATH=$ANDROID/linux-msm-rhod-nand

# TOOLCHAIN_PATH -- directory containing the arm toolchain

TOOLCHAIN_PATH=$ANDROID/prebuilt/linux-x86/toolchain/arm-eabi-4.4.0/bin/arm-eabi

# MODULES_PATH -- directory for modules

MODULES_PATH=$TINBOOT/modules

#-------------------------------------------------------------------------



# do some cleanup before building if zImage already exists

echo "Removing previous zImage"

if [ -f $TINBOOT/kernel/zImage ] ; then

     rm $TINBOOT/kernel/zImage

fi



# build kernel and modules

#export ARCH=arm

cd $KERNEL_PATH

make clean

make ARCH=arm htc_msm_nand_defconfig

make ARCH=arm CROSS_COMPILE=$TOOLCHAIN_PATH- INSTALL_MOD_PATH=$MODULES_PATH zImage modules modules_install

[ $? -eq 0 ] || fail "Kernel compilation failure"



# copy all drivers to single level directory for compression

cd $MODULES_PATH

KERNMODS=$(find -name "*.ko")

if [ ! -d $ANDROID/androidupdate/data/modules ] ; then

     mkdir -p $ANDROID/modules

fi

for i in $KERNMODS ; do

     cp $i $ANDROID/modules

done



# make squash files

KER_VER="$(cat $KERNEL_PATH/include/config/kernel.release)"

mkdir -p $ANDROID/androidupdate/system/lib/modules

mksquashfs $ANDROID/modules $ANDROID/androidupdate/system/lib/modules/$KER_VER.sqsh



# compress modules to a tar.gz and move to output

echo "Outputting modules to Desktop"

cd $ANDROID/androidupdate

tar -cvzf $ANDROID/androidupdate.tgz data system

rm -rf $ANDROID/modules

rm -rf $ANDROID/androidupdate/system/lib/modules



# clean up modules

rm -Rf $MODULES_PATH

mkdir $MODULES_PATH

rm -Rf $KERNEL_PATH/lib/modules



# move new kernel

mv $KERNEL_PATH/arch/arm/boot/zImage $TINBOOT/kernel



#-----start build XIP-----

# remove old XIP if it exists

cd $TINBOOT

if [  -f xip/rhod ]; then

    rm xip/rhod

    echo Deleting old XIP

fi



# make XIP folder if it does not exist

if [ ! -d "$TINBOOT/xip" ] ; then

mkdir $TINBOOT/xip

fi



# compile tinboot XIP

echo "Start compiling tinboot/xip for rhod....."

$TOOLCHAIN_PATH-as  tinboot/tinboot2.S -o tinboot.o --defsym rhod=1 --defsym MTYPE=2292

$TOOLCHAIN_PATH-objcopy tinboot.o -O binary tinbootxip

mv tinbootxip xip/rhod



# cleanup tinboot

rm tinboot.o



# make sure XIP build was successful

if [ ! -f xip/rhod ] ; then

    fail "XIP build failed. XIP file was not found."

fi



#-----start build NBH-----

# clean up previous log file

if [ -e "tools/log" ]; then

rm tools/log

fi



# compile tinboot

cp tools/rhod_payload2 os.nb.payload



echo "Inserting tinboot into payload"

wine tools/osnbtool -c os.nb.payload 1 xip/rhod >> tools/log

mv os.nb.payload.NEW os.nb.payload >> tools/log



echo "Inserting blank imgfs into payload"

wine tools/ImgfsToNb.exe  tools/imgfs.bin os.nb.payload os-new.nb.payload >> tools/log



echo "Creating os.nb portion of nbh"

tools/nbmerge < os-new.nb.payload > os-new.nb



echo "Creating NBH"

wine tools/yang.exe -F RHODIMG.NBH -f os-new.nb -t 0x400 -s 64 -d RHOD****0 -c 11111111 -v Tinboot -l WWE >> tools/log



# clean up payloads

mv RHODIMG.NBH $ANDROID/RHODIMG.NBH

rm os.nb.payload

rm os-new.nb.payload

rm os-new.nb



# clean up tinboot directory

rm -rf xip

What the heck is zImage, modules, tinboot, rootfs, initrd, and System.ext2?
This section will be updated slowly as I have time.

The zImage is the kernel (which is required to run the operating system). We currently use the linux kernel 2.6.27.46.

The modules are your drivers. These are a matched pair with the zImage/kernel. It is also important to clean up the modules periodically when building the kernel/modules or you will start running out of space :)

Awesome explanation of the kernel and its modules here: http://www.howtogeek.com/howto/31632...at-does-it-do/

Tinboot is the bootloader for running Android from the NAND. Currently, we put initrd.gz and the zImage on the NAND while the zImage, rootfs, and System.ext2 are pulled from the SD Card.

RootFS is a special type of ramfs file system. It basically lets us use part of the memory as a partition for caching files. This gives us a nice performance boost :)

Initrd is an initial RAM disk that is temporary and used only for booting. Generally it uses a slightly different scheme than rootfs does.

System.ext2 is your file system. This is technically your hard drive where all of the Android files and applications are stored. It uses the ext2 file system which is an extended file system designed specifically for linux.

Re: How To Compile Your Own Kernels, Modules, Tinboot (NAND boot), and More for Begin

Thanks for the info on this Nate! :occasion5:

Re: How To Compile Your Own Kernels, Modules, Tinboot (NAND boot), and More for Begin

I've compiled for HARET before once. This really helps though.

Thanks!

Re: How To Compile Your Own Kernels, Modules, Tinboot (NAND boot), and More for Begin

Can't seem to get past this step:

Code:

make ARCH=arm linux-msm-nand-defconfig

What I see:

Code:

~/Desktop/Android/linux-msm-rhod-nand$ make ARCH=arm linux-msm-nand-defconfig

  HOSTCC  scripts/basic/fixdep

scripts/basic/fixdep.c: In function ‘traps’:

scripts/basic/fixdep.c:377: warning: dereferencing type-punned pointer will break strict-aliasing rules

scripts/basic/fixdep.c:379: warning: dereferencing type-punned pointer will break strict-aliasing rules

  HOSTCC  scripts/basic/docproc

make[1]: *** No rule to make target `linux-msm-nand-defconfig'.  Stop.

make: *** [linux-msm-nand-defconfig] Error 2

Everything else went good. Any Ideas?

Thanks!

Re: How To Compile Your Own Kernels, Modules, Tinboot (NAND boot), and More for Begin

Quote:

Originally Posted by Lmiller1708 (Post 1982356)

Can't seem to get past this step:

Code:

make ARCH=arm linux-msm-nand-defconfig

What I see:

Code:

~/Desktop/Android/linux-msm-rhod-nand$ make ARCH=arm linux-msm-nand-defconfig

  HOSTCC  scripts/basic/fixdep

scripts/basic/fixdep.c: In function ‘traps’:

scripts/basic/fixdep.c:377: warning: dereferencing type-punned pointer will break strict-aliasing rules

scripts/basic/fixdep.c:379: warning: dereferencing type-punned pointer will break strict-aliasing rules

  HOSTCC  scripts/basic/docproc

make[1]: *** No rule to make target `linux-msm-nand-defconfig'.  Stop.

make: *** [linux-msm-nand-defconfig] Error 2

Everything else went good. Any Ideas?

Thanks!

Nice catch. It should have been htc_msm_nand_defconfig. Updating now :)

Re: How To Compile Your Own Kernels, Modules, Tinboot (NAND boot), and More for Begin

Quote:

Originally Posted by natemcnutty (Post 1982438)

Nice catch. It should have been htc_msm_nand_defconfig. Updating now :)

That seemed to work.
But the Next Step:

Code:

make ARCH=arm  CROSS_COMPILE=~/android/prebuilt/linux-x86/toolchain/arm-eabi-4.4.0/bin/arm-eabi- INSTALL_MOD_PATH=~/android/modules zImage modules modules_install

I get this:

Code:

make: /home/lance/android/prebuilt/linux-x86/toolchain/arm-eabi-4.4.0/bin/arm-eabi-gcc: Command not found

  CHK     include/linux/version.h

make[1]: `include/asm-arm/mach-types.h' is up to date.

  CHK     include/linux/utsrelease.h

  SYMLINK include/asm -> include/asm-arm

ln: creating symbolic link `include/asm': Operation not permitted

make: *** [include/asm] Error 1

Re: How To Compile Your Own Kernels, Modules, Tinboot (NAND boot), and More for Begin

Quote:

Originally Posted by Lmiller1708 (Post 1982521)

That seemed to work.
But the Next Step:

Code:

make ARCH=arm  CROSS_COMPILE=~/android/prebuilt/linux-x86/toolchain/arm-eabi-4.4.0/bin/arm-eabi- INSTALL_MOD_PATH=~/android/modules zImage modules modules_install

I get this:

Code:

make: /home/lance/android/prebuilt/linux-x86/toolchain/arm-eabi-4.4.0/bin/arm-eabi-gcc: Command not found

  CHK     include/linux/version.h

make[1]: `include/asm-arm/mach-types.h' is up to date.

  CHK     include/linux/utsrelease.h

  SYMLINK include/asm -> include/asm-arm

ln: creating symbolic link `include/asm': Operation not permitted

make: *** [include/asm] Error 1

Try make clean, then do the defconfig one more time, then the last make. Also, double check and make sure the arm-eabi-gcc is there. If not, do a git pull in the prebuilt directory.

Re: How To Compile Your Own Kernels, Modules, Tinboot (NAND boot), and More for Begin

FYI this "how to" is ACL approved :-)

:thumbright:

Re: How To Compile Your Own Kernels, Modules, Tinboot (NAND boot), and More for Begin

This is a GREAT thread, thanks Nate