Why do so many non blacks have problems with black colleges, NAACP, UNCF, and other black institutions.?

So many times I have heard people complain about affirmative action, black colleges, BET, and UNCF. I don't know if it is more that these entities are misunderstood, or people really think they are racist. People have to know that all of these entities were formed to give African Americans a chance at a time when we had none. None of these entities say kill whitey or nothing of that sort. So what is the big deal with this? Is it just cluelessness or ignorance. I really don't know. Anybody with half a brain should know that they are not a source of racism, but a source of empowerment. If we need to get rid of BET, or UNCF, should we also get rid of the WE channel and scholarship that are formed for women?