Mr Peters has also foreshadowed fresh support to Ukraine as the embattled country attempts to fight off the Russian invasion.
Mr Peters' call is at odds with the pre-election wishes of the National and Act parties; however, the leader of the smallest coalition partner, New Zealand First, has won the day.
"That's not a good idea. It's better to jaw, jaw, jaw than war, war, war and we need to talk to the Russians," he told AAP.
"That sort of knee-jerk reaction is not diplomacy."
Mr Peters' decision rebukes Prime Minister Christopher Luxon and Act leader David Seymour, who both argued to expel Ambassador Georgii Zuev for Russia's illegal invasion.
The expulsion was not included in coalition deals struck between the parties, leaving it up for debate.
In an interview with AAP, Mr Peters said he was "very concerned" about the war's trajectory, but an expulsion wouldn't help.
"There'll be a different regime sometime in the future and the fact that we have got everything in situ after ... upgrading our facilities in Moscow is something that we should consider seriously," he said.
"Rather than a rash reaction that says somehow we're going to advantage our helping there to be a peaceful conclusion in Ukraine by booting the ambassador out."
New Zealand has levied a vast range of sanctions and travel bans against Russia since its ground offensive in February 2022.
It has also made several contributions to Ukraine, including thousands of pieces of non-lethal defence equipment, training Ukrainian troops, $22 million worth of military funding, and $17 million in humanitarian support.
Still, those contributions make New Zealand among the smallest donors per capita in the developed world.
Mr Peters said "you might see some developments very soon" of fresh support.
"There are some utilities which may make much more sense in their hands to be used now, than to be in our hands that are never going to be used in the future," he said.
Mr Peters, 78, is in his third stint as foreign minister, serving in the post under Labour prime ministers Helen Clark and Jacinda Ardern in previous coalitions.
Ironically, he finds himself responsible for one of those old bosses, Ms Ardern, who took a non-paid post as New Zealand's special envoy for the Christchurch Call, a pact to eliminate online extremism involving both countries and tech companies.
The Christchurch Call was founded by New Zealand and France following the 2019 terror attack on two mosques in the city which killed 51 people.
Ms Ardern's role was due to be reviewed at the end of the year, but Mr Peters said he was yet to make a call on continuing the project or her involvement.
"What I'd want to know is if that project has a long term sincerity and integrity. How's it going? What's it achieved? I'd want a readout of that," he said.
Ms Ardern's office did not respond to comment about her future intentions.
Mr Peters has fallen out with Ms Ardern since his party's election loss in 2020, and was irascible at her mention.
"She was never my boss, with the greatest respect," he said.
"I was the guy that gave her the chance which meant that she became somebody, and nobody would have ever heard of again had I not given her the chance."
Mr Peters also told AAP he didn't ask for the foreign ministry, joking that he ended up in the job over National's foreign affairs spokesman Gerry Brownlee, now parliament's speaker, because he "didn't want to be speaker".
"It was embryonically thought about and matured without a lot of discussion," he said.
