Maybe in the back of his mind he was thinking how Sonny was killed, and even though he had a son, what if, heaven forbid, he was killed. Mary surely wouldn't take over the family. Perhaps it was just the more sons the better. There would always be male heritage to take over the family business.