I am the type of person who normally does not like to talk on the phone. I get very agitated when I make a one on one phone call to someone and they put me on speakerphone. For one thing, you cannot hear someone clearly when they use the speakerphone and it picks up a lot of background noise. Also, the conversation can be heard by anyone near the person you are talking to. I don't like that invasion of privacy. I used to work in an office where lawyers close to my desk would have conversations with their client on the speakerphone. You could listen to the whole conversation and I don't think that is appropriate.
As soon as someone answers their phone, I immediately know when I am on speakerphone and I will always ask that they take me off it. I don't care who I am talking to. My sister-in-law used to always call me on speakerphone until one day I said to take me off it. Maybe that's why she doesn't call me much anymore. The only time that I will talk on speakerphone is when there is a need for the conversation to be heard by more than one person like in a business meeting. Otherwise, I think that if a person wants to speak with you on the speakerphone that they should ask for your permission to do so out of courtesy.