I don't think that exile would have been enough, only because of the outburst in the boathouse. As Turnbull said, the boathouse scene confirmed that not only did Fredo know, but he also had resentment over being "passed over". IMO the resentment is what did him in, not the act of treachery in and of itself, because the resentment showed that he would never be happy.
Let's see that he's exiled and cut off from all the family business. Nobody directly in the Family deals with him. He's given some "Mickey Mouse nightclub" to manage to earn a living, and told pretty much to never contact anyone, least of all Michael. Who's to say that some other conniving gangster wouldn't approach Fredo down the road with another deal and promise him something of his own? Who's to say he wouldn't take that offer, either growing the balls to try and off Michael himself (not likely, but still), or snitching to the police, or giving any information at all that could be of some possible use against Michael and the Corleone Family, even if it's outdated.
Fredo HAD to die, because he showed that he would never be happy unless he was the Don, and that meant he could always face the temptation of yet another betrayal because he would always resent the fact that his "kid brother" was made head of the Family.
Who cares if Fredo is not happy? That doesn't make him a threat. Especially if he's exiled. He would have no information to provide and no physical way to get at Michael. Mafia Dons make people unhappy all the time. Where's the danger Fredo posed to Michael?
Yes, he could have gone to the police. But it's not like Michael killed him right after the boathouse. He allowed Fredo to live as long as Mama was alive, which could have been years, and even permitted Fredo to visit the compound. None of the supposed dangers came to pass.
If Michael at his most paranoid didn't consider Fredo a threat, it will take more than vague hypotheticals to make me believe he was.