Tully, 48, was detained after a seven-hour manhunt 15km from the Ashburton Winz centre where it's alleged he gunned down three workers.
The sawn-off shotgun police believe he used in the shootings was found "very close" to Tully when he was brought down by police dogs, the source said.
Witnesses have told how the gunman is alleged to have broken down the gun after Monday morning's bloodbath, placed it inside a backpack, and cycled off.
The following day, Detective Inspector Tom Fitzgerald, Canterbury district investigations manager, confirmed that Tully did not have a firearms licence.
But it wasn't until the following day that police released a statement saying they had found the alleged firearm.
Asked why it took police 48 hours to tell the public they knew where the alleged gun was, Mr Fitzgerald replied: "As per the media release, that's all I'm saying on the topic."
Mr Fitzgerald did, however, confirm last night that police were still in the dark on where the gunman had got it.
"Inquiries are continuing to establish where he obtained the gun," he said.
Speculation is rife in the Mid-Canterbury town that he either stole the weapon, or bought it from a gang on the black market.
A memorial service for Peggy Noble is planned for 11am on Monday.
- Ashburton Guardian