Yes, affinity is neither fish nor fowl. Two little stories because that's how I roll.
Forty-plus years ago I lived in Georgia (the state in America) and the subject of integration came up in conversation. A man said to me (approximate quote), "We can't have those children going to school together, they'll think they're the same." He knew exactly what familiarity could do and his motivation for wanting to prevent it was clearly racist.
Two days ago as I was putting my groceries in my truck a guy came walking up to me (panhandler alert!). He was young (relatively) and able looking, not the type I usually give money to. He must of seen my license plate which has the USMC bird on a ball insignia. He showed me a picture of his Marine Corps ID card from when he was on active duty and pulled back his shirt to show me his Marine Corps tat. He said he was recently divorced (she must have had the best lawyer) and was trying to get to San Diego with his dog. His stuff was in a shopping cart. I gave him $20. He exclaimed, "Oh my God, thank you!" I stuck out my hand to shake his and said "Semper Fi." Another affinity, the brotherhood of Marines (it includes sister Marines). If not for that he wouldn't have gotten a dime. My natural bias is to help old people, especially women. Obvious bias at work in this.
Affinity, or lack of it, comes in many flavors. I, like everyone, have them and they enable bias. The best we can do, in my opinion, is to keep them wholesome. I know about some of my biases and no doubt have some below the surface. Probably because I wish they were not there.