Many scenes end up on the cutting room floor.
But, as the novel explains, it was only Luca who could make Vito feel nervous because Luca was an uncontrollable force. Luca let Vito control him. Also, Luca's reputation was well known in the underworld and allied community, so his presence might make some people uncomfortable. Afterall, it was a celebration.
More detail: Tom tells Vito, "He didn't expect to be invited..." I took that to mean that Vito's nervousness about having Luca around lingered until it meant that Luca's invitation was a last-minute decision.
Probably Vito felt a) Luca's fearsome appearance would make other guests nervous; and b) his presence would be a reminder to some people that the Corleone empire rested on the kind of brute force that Luca represented.
