I have been messing with this too since Pibe linked to this thread from another thread.
I don't think the issue is so much with the .3gp extension. I have The Core Media Player installed on my 700wx and I have the .3gp checked off in the "file extensions" section in settings. The issue seems to be with the "rtsp//:" protocol.
When I click on a link on the mobile youtube page PIE pops a window that says:
"Running a system command on this item might be unsafe. Do you wish to continue?"
I click "yes"
Then Core opens and pops a window that says:
"RTSP protocol not supported!"
RTSP stands for "Real Time Streaming Protocol" and the player that the device wants to use must support it. Evidently Core does not unless they have an add-on or update. I don't know about WMP.
Here is an RTSP FAQ:
http://www.rtsp.org/2001/faq.html