I believe it was cut and dry for Michael.

The boathouse confrontation did nothing to seal Fredo's fate. It was already decided. As soon as Michael found out that Fredo was the traitor (in Havana,) he was a walking dead man. Nothing, with the exception of Vito coming back from the dead or Jesus coming off the cross, was going to stop Michael.

Maybe not even then.