"We're now in a much happier space, where most central banks feel we're back on top of inflation, [we're] not there yet."
"But inflation expectations have been the big concern, the more people think inflation will rise next year, the more inflation will rise next year."
The RBNZ has held the official cash rate (OCR) unchanged since May last year, and indicated that cuts may start from early next year, although financial markets have been pricing in cuts from August this year.
Orr said the New Zealand economy was tracking to the RBNZ's expectations, and the bigger than expected contraction in growth at the end of the last year and dip back into recession was part of the necessary readjustment.
"That means that we are on track to getting inflation back into the target band."
"Aggregate demand is slowing, core inflation and pressures are coming off, and inflation expectations are coming back, so we hope that we can see low and stable inflation on the horizon again and that would mean more normalised interest rates on the horizon."
Regulating banking services
Meanwhile, Orr is not convinced the Commerce Commission's recommendation to loosen banking rules and supervision would necessarily deliver greater competition in the personal banking market.
The commission's draft market study last week pointed to tight banking rules about capital levels and systems as barriers to competition and innovation needed to break the grip the big four Australian banks have.
The RBNZ is the regulator of the banking sector and has been requiring banks - big and small - to increase their capital, and to get systems ready for a deposit insurance scheme, which smaller entities have complained puts them at a further disadvantage.
"The main work we do is not going to shift the competition dial at all."
He said the RBNZ could not let lots of small companies fail "in the hope that it creates competition".
It was instead looking at allowing greater access to its settlements system, Orr said, which allowed selected financial institutions to settle transactions in real time, but that would not bring much change.
He said open banking would be the key change to disrupting the banking sector and boosting competition.
"That is critical to competition, people being able to take their bank number wherever they like and more smoothly shift between bank services and allow fintech (financial technology) in there ... those are two of the game-changers."