I think the decision had to do with Geary's brashness in Michael's office. Geary, much like Woltz, miscalculated that the long arm of the Corleone's power could not reach his own world. Perhaps this lead to his arrogance and unnecessary harshness in Michael's office. Geary's insults and attempts at extorting Michael were probably a factor in the decision to not-so-subtly say, "See this dead hooker. Keep it up or you'll be next."

I believe that if Geary had been civil and polite to the Corleones, they would have muscled him in a different way. But Geary opened the can of whoop-ass that was eventually brought down against him.