Humanity is not perfect, and never will be. If you expect it to be so, you'll be SORELY disappointed. As long as we're different and capable of violence, there's going to be some of it.
I think you're exaggerating it though. Humans, like most animals, usually try to avoid conflicts. Most people i know would rather argue (more or less energetically) than fight. Not to mention that most fights are preceded by lengthy and/or intense arguments, while others are sparked by hatred and anger. Saying "fuck you and your favourite colour" to someone and then leaving just isn't enough every time. If you say you've never wanted to hit someone, then you're either a true buddhist or a liar. People who actually hurt others impulsively are what we know as criminals, and shouldn't be the baseline of humanity.