Very simple. It's what Vito would have done had he been in Barzini's situation. (In the book it's explained that Vito did this to the Maranzano character years ago).

Vito wouldn't have been able to predict all the details but with 30 yrs + experience he would have known that like most such attempts, the murder of Michael could not have been easily accomplished without betrayal. It was just a question of who.

Also remember that although it was unspoken, everyone in the family blamed Carlo for Sonny's death though they pretended not to do so. In that instance they had been fooled once. They weren't going to be fooled again by Barzini.

I don't know if it's a Mafia custom other than pure pragmatism. Your "friends" are the ones who set you up/kill you because they are the one who can most easily get you relaxed, alone and unguarded. As Vito said, it's an old trick.