teddat,
Try moving your finger slower. If you are swyping really fast then you may not be applying enough force as it would then move the phone but when your holding it with the other hand you can apply more force as you are holding it and a faster swype does then work. Hope this is all it is and you don't have a bad handset.