The olive oil business was the "legitimate" cover for the Corleone family rackets: gambling and unions. When Michael said the family was leaving the olive oil business, he meant that they were quitting New York and relocating their operations to Nevada to the (legitimate)casino business. But, as we saw in GFII, complete "legitimacy" had eluded Michael.