I wonder if Fredo really had men of his own or if Fredo's men were really just Corleone men who would do reasonable Fredo biddings- and for anything other than reasonable biddings they'd consult Michael, Neri or Rocco. I'm not sure I could see Fredo's men being comfortable in assisting in the assassination of the don- on Fredo's order.
