Get a memory card to begin with if you haven't already done so. Most apps can be installed to the card which will keep your limited phone memory from filling up too fast. Then get the SPB backup like johncal mentions.
Most programs and tweaks come in cab files. The file can be copied to your phone (you can create a folder on the memory card to temporarily store the cabs) although this is not always necessary. Then click on the cab. It will start the install. There will usually be a warning message about 'unknown publisher'. Tell it to install anyway. (You will be doing all this in File Manager.) It may ask you where you want to install it to. Usually you will select the memory card for the reason stated above. If it doesn't ask, don't worry about it.
When it completes, close file manager and look in your Applications list. It should be there.
Tweaks, colors and themes will usually require a reboot. If it doesn't ask you to, but you don't see any changes do it anyway.
Sometimes an app will come in exe format. Those are generally installed after you have connected your phone to a PC and are running Active Sync (older) or Windows Mobile Device Center (Vista/Win 7). You run the executable on your PC and select the option to install on the mobile device.
Most things can be uninstalled but it is good to ask first if you are concerned.
At worst you may have to hard reset if something really messes it up, which puts it back to the way it was out of the box. I've installed and removed a ton of stuff and I haven't had to do that (yet).
OK, I see you have the SD card. Step one done!
