In my own personal experience, I've noticed that video games tend to be detrimental to a person's character and efficiency, however I wouldn't say that they are completely harmful overall. I've seen in myself and multiple peers that a moderate indulgence in video games usually isn't too difficult on an individual's time and work ethic, but there is a border between moderate and sever indulgence which spells the difference between "Normal Life" and "Isolationist Life."
Purely in myself (and thus probably not completely extendable to others) I have noticed that video games eat away a ton of my time, and more and more I find myself having to set alarms and timers in order to stop gaming in time to complete a paper or such. It's almost impossible to keep up with gaming and my school work at the same time without some sort of give on either end. On more than one occasion I have assigned higher priority to the former, and now I've become pretty desensitized to the decision.
A side-effect of the little time I allow myself for anything outside of gaming and school work is that I have very little time to go and hang out with friends. What I feel to be a result of that is my own anti-social behaviour. I don't mean that people frighten me, I mean that people tend to annoy me. I would much rather be on my own, and I've gone so far as to ask some of my teachers to exclude me from group activities. If that truly is an effect of a severe level of gaming, as I suspect it is, then you should see some amount of isolationism in many people partaking in a similar amount of gaming.
Veering away from myself, I've noticed that gaming tends to be an escape from reality for a lot of people in difficult situations. Of course, one could argue that video games are simply the means to an end for these types of people. As such, I won't do this aspect of the discussion any justice besides a brief mention of it. If anyone wants to delve deeper, feel free.
Although this has been almost exclusively pertaining to myself, I think it's fairly safe to assume that we should be able to see a similar trend of some magnitude (be that .005 or 2) in most people who have crossed that border between moderate and sever gaming. This is, without a doubt, harmful to society. A degradation of responsibility affects every aspect of a person's life, from finances to sanitation. To be completely honest, I haven't thought about the effects of increased isolationism in people's lives beyond the simple fact that it can make it very difficult to actually enter into society. In any case, if either of these effects show to any degree in people that partake in a similar amount of gaming to myself (and many other "gamers") then you could say that video games are harmful to society.