I think that Roth viewed himself as being in a long quiet defensive war against the Corleone Family. In a period of less than a decade the Corleones have murdered Roth's best friend, taken over several Roth associated or owned hotels and casinos in Nevada and made moves into Roth's Cuban domains.

Roth evidently doesn't have the muscle to openly resist these things.

But if he can get the Rosatos to kill Frankie, he can hopefully draw Michael into a long war in NY. This weakens the Corleones and could give Roth the opening to save his own interests. As a bonus Roth might even be able to convince the Corleone NY branch that Michael had turned against them and start a family civil war.

Also despite Roth's protestations of business above all he really did view Moe Green as a friend and may have been waiting patiently for the chance to pay Michael back by taking out someone near and dear to him. This idea makes more sense when we remember that the Pentangeli character is really supposed to be Clemenza.