I'm not entirely sure I understand the problem, either. But I guess I can just reiterate what Petit said. Try to just focus on the music. Look at each song as only that: an individual song, unattached to a person or fanbase or even an album.

I like Kuu just fine, although I don't follow her like I do Ayu or BoA. I loved the 12 Singles, and I have the music from each of her previous albums. But I only really pay attention to the ones I like. It doesn't make me think any more or any less of her. I just like Kumi has different taste in music than I do in a lot of regards! But then, in other places, we "get along" just dandy.

At the end of it all, you can respect someone without being a die-hard fan.

And if all other advice fails, think of it this way: the less aritists you follow, the happier your wallet.