I guess I'm spoiled as all my contacts were on an Exchange Server, so I exported them all from Outlook, and imported them to Google.
I've not used the "phone" contacts.
Good luck!
(I do like the Google integration, and it even keeps track of the "favorites" from the people app, so it's a win
).