To me, the telling part is what comes next: "Do you spend time with your family? Good. Because a man who doesn't spend time with his family is not a real man." It just so happens that Sonny enters the room on the last sentence.

What Vito's saying is: "Ok, you philander, you run around with a Hollywood crowd, you screw around with the studio boss's protégé. It's all understandable, part of the human condition. But the bottom line is that you are anchored in your family--that's what counts."

I bet if Johnny hadn't answered, "Sure, I do" to Vito's question, Vito's help with Woltz wouldn't have been so forthcoming.

Also couple that look he gave Sonny when Sonny entered the office with his later admonition to Sonny at the end of the Sol meeting: "What'sa matter with you? I think all that comedy you're playing with that young girl is making your brain soft." He's giving Sonny a similar message: "Play around if you have to. But never let it get in the way of business."