Criminals will always have guns, weather they steal them, make them, find them, etc. They will always have guns. When you try and take them away, you are taking them away from the citizens giving the criminals the upper hand.

Let's say you are going to rob a house. If you know, or at least think there is a gun in that house, are you going to go in? Maybe you still will, but you will think about it A LOT more than if you knew there wasn't any guns in that house.