I think the only time you know Fredo is carrying a gun is when he is at work, at the Genco Olive Oil importers

It should be assumed, given the Don's connections at the time a weapon's beef wouldn't mean much, though the tables get turned when a police captain enters the mix.

Guns are around all the time, and none of the time; the fact that Fredo has a .38 Special when he is driving his father one day shouldn't mean much of anything.

Being "Muscle" means you are a gunner when need be, but not just because you are a book-keeper, runner, brothel manager, pit boss, etc who might have a piece around.

The "Muscle end of the family" are the debt collectors, hit men, bouncers at clubs, guards when you need to go to the mattresses, shake-down men etc

Carrying a gun from time to time doesn't put Fredou in that club.