Sergeant Karl Hemmingsen, of Dunedin, said the man entered the Gun City store in Cumberland St about midday, and was questioned by a staff member about possible shoplifting.
It appeared he had second thoughts about his actions and he fled the scene.
Sgt Hemmingsen said the man did not escape with any ammunition.
He was arrested 10 minutes later, nearby in Princes St, and taken to the Dunedin Central Police station for questioning.
"We’ll make a decision on charging once we’ve got the full story."
A shop spokesman said no-one was threatened or injured during the incident.