The officer from Whakatane admitted firing an arrow at 4-year-old bull mastiff Zeta after the dog wandered on to his property last November.
The arrow struck Zeta in the chest, causing serious injury.
Although the officer was not stood down, police confirmed a criminal investigation would take place.
A police spokeswoman last week said the officer would not have any criminal charges laid against him as the case did not reach the threshold for a criminal prosecution.
There has been strong public reaction since the decision was announced.
More than 100 people, many with pet dogs by their side, took part in a hikoi against the decision through Whakatane this morning.
The public march began at The Warehouse on Kakahoroa Dr, and ended at the Whakatane police station.
Event organiser Wharenui Tuna said he had been following the story since last year and noticed there was strong public opinion on social media.
"My opinion is that it is totally unfair, unbalanced and the general public feel the same way too.
"We want them to reverse their decision and prosecute ... At the very least, the officer should be stood down."
Mr Tuna said he was "very disappointed" a representative from the police did not meet them at the end of their march to listen to their concerns.
"I'm very disappointed. One of the things we were looking for with the hikoi was to create an honest and open relationship with the police."
Meanwhile, an online petition has gained almost 1400 signatures from members of the public who are calling on police to revisit their decision.
Under the Animal Welfare Amendment Act 2010, it is an offence to willfully or recklessly ill-treat an animal involving prolonged pain and suffering and the need for vet treatment.
SPCA chief executive Ric Odom said while it was not acceptable shoot an animal with a crossbow or a firearm, the decision on whether or not to charge the officer had to be made by police.
Mr Odom said given public reaction, he would be surprised if police were not revisiting the case.
"But the basis to charge or not should be based on the evidence they have received and uncovered and the circumstances surrounding it.
"I can only assume they have gone through this process, and for reasons best known to themselves, have clearly come to the conclusion that it is not in the public interest to charge someone for this offence."
Mr Odom said although it was discussed, the SPCA would not be conducting its own investigation into the incident.
"We did think about that here at the national level ... but it would be most unusual for us to do that.
"The police have the same powers under the Animal Welfare Act that we do, and they've gone through the same process we would have gone through."
A police spokeswoman today said police were aware a public march had been organised, but they were unable to comment on whether police would revisit the case in light of public outrage.
"The inquiry is still ongoing and we have nothing further to add at this stage," she said.
Whakatane police could not immediately verify how many people took part in the march.
By Nikki Papatsoumas of NZME. News Service
