There are some threads on this over at PDAPhoneHome.com
http://pdaphonehome.com/forums/ppc-6700-xv6700/
I believe one of the better suggestions was using a piece of an eraser from a mechanical pencil.
And everyone suggested using superglue
applied to the rubber NOT into the hole.
Good luck.