You are not permitted to download, save or email this image. Visit image gallery to purchase the image.
Speaking to Three's The Nation, Andrew Little said he was going to approach the issue "very sensibly".
"It's actually not that hard if we choose to resource it properly."
Although some hardened criminals needed to stay locked up because they were a danger to society, a "whole chunk" of prisoners were there because they were battling other issues which had driven their offending, he said.
Too many people with mental health problems and other issues weren't getting the help they needed while in prison, Little said, and so were unable to meet the conditions they had to get parole.
Ensuring they were properly rehabilitated would make it easier for ex-prisoners to integrate back into society and reduce reoffending.
He also revealed he would review how sentencing and bail was being managed by the courts.
Part of the reason our prison population was so high was because we were jailing people for longer, he said.
Little said the issue was not necessarily with the legislation, but instead more likely stemmed from how it was being applied and enforced.
"What we do have to do is get some consistency."
Although judges were doing a good job and interpreting the law the best they could, stronger guidelines were needed.
One way to do this would be by establishing a sentencing council, he said.
Little would also look into how bail was being administered. He questioned whether it was reasonable to lock up many people who had been charged but were yet to be tried.
While there was "no question" the safety of the community needed to come first, it needed to be balanced against the actual risk offenders awaiting trial posed to the public.
He said it was "thoroughly enjoying" his role as Minister of Justice, Minister for Courts, Minister for Treaty of Waitangi Negotiations and the Minister responsible for the GCSB, NZSIS and Pike River re-entry.
"It's been a fascinating year."
However, he had also "thoroughly enjoyed every minute of being Leader of the Opposition".