Why does Clemenza and Sonny keep reminding Michael to "drop the gun" right away after he whacks Sollozzo and the police captain? What's so important about dropping the gun right away?
Because if anyone catches you with the gun on you, you're screwed. Leave it there you can't be linked to the crime.