Mr Tall, of Cromwell, died on the Old Man Range, near Cromwell on Thursday night.
The 61-year-old was hunting with a companion but was alone when his rifle "discharged accidentally, striking him in the abdomen area'', Detective Sergeant Brian Cameron, of Otago Lakes Central CIB, said yesterday.
Wayne Tall, of Dunedin, said his elder brother was a "very experienced hunter''.
"He loved hunting and it was one of his favourite pastimes. He was very experienced in hunting and in handling guns.''
Mr Tall said he had been "racking my brain to work out how this could've happened; we're in shock and can't explain it.''
His brother was a real outdoorsman, who enjoyed fishing and boating as well as hunting.
He had moved to Cromwell from Dunedin with his family a year or two ago and scaled back to working four days a week.
Mr Tall was on a day off when the accident happened.
"Gary was an engineer who could build anything and we prided ourselves on fixing anything,'' he said.
Gary "enjoyed a good laugh'', was close to his family and enjoyed spending quality time with wife Jan, children Andrew and Krystan and other family members.
Mr Tall hoped the police investigation would "answer all of our questions'' and reveal the cause of the accident.
Det Sgt Cameron said he did not want to speculate at this stage on the cause of the accident.
"Obviously, we're in the process of looking at and eliminating possible scenarios as part of our inquiry.''
Until that process was completed and the postmortem examination had been carried out, any suggestions would be speculation.
The rifle would be tested by the police armourer to see if it had any defects, which might have been a factor in the incident, he said.
Police were satisfied there were no suspicious circumstances.
The accident happened at a remote site, near Duffers Saddle, and cellphone coverage was patchy.
Wayne Tall's hunting companion put an injured Mr Tall into his vehicle and drove to the road through the Nevis Valley to meet emergency services staff, Det Sgt Cameron said.
Police and ambulance staff performed CPR on Mr Tall, but were unable to revive him.
Victim Support was helping Mr Tall's family and his hunting companion.
A karakia would be held at the site, Det Sgt Cameron said.
