Most of my family such as my parents, sisters, brothers etc I speak to because I still love them.
But some of my other family such as an aunt, uncle etc I stopped speaking too because I hate them and hate being around them.
They cause more trouble to me and it's not worth being around them just because they are considered by law to be family.
Nobody picks the family they have and you can choose to not speak to that family if you do desire.
If a family member or family members are causing you trouble in your life then you can and should shut them out of your life and not deal with them anymore.