He abused Connie because he was bitter that the Corleones did not make him an important part of the family and instead gave him just a living.

There is a deleted scene where Connie and Carlo are fighting in the Corleone house and Mama complains to Vito about it and Vito tells her not to interfere. Then later on, after Vito is shot, the family is sitting around for dinner and Carlo tells Connie to shut up at the table and Santino almost jumps over the table telling Carlo "Hey, don't you ever tell her to shut up again" (Another classic Sonny scene ) and Mama Corloene cuts in and says "Santino, don't interfere."

The "don't interfere" by Vito and then the "don't interfere" by Mama were, in Carlo's eyes, a license to treat Connie however way he wanted to. So he abused her because deep down he knew that what the Corloene's had given him in within the family business was menial.

I think that in the novel there is even a part where Puzo tells us that Carlo would get abused verbally by some of his friends because he didn't have a "real position" in the Corleone family, so he went home and took it out on Connie. (Olivant or TB should be able to confirm or deny this for me).