I think the only reason it is even an issue is that the line is spoken off-screen. We don't see Clemenza b/c the camera is on the fish & vest. Don't forget - Clemenza was also sitting next to Sonny and it's very clear that the lines spoken have the tone of Clemenza's voice.