This is not illegal, in some cases you need to inform them. This is very valuable as I record conversations with customer service (since they always lie and never follow though).
most states require only 1 party consent. This means only either caller or person being called needs to know about it to be legal. Some states require both parties to know about it.
List of 1 party/2party states. Under "State Laws (Table)" here.
http://www.callcorder.com/phone-reco...aw-america.htm