You are not permitted to download, save or email this image. Visit image gallery to purchase the image.
Peters conceded in a short speech to party faithful in Russell.
He thanked voters, volunteers and NZ First staff around New Zealand.
"We committed ourselves three year ago to be a constructive partner with government," he said.
NZ First had sought to "provide certainty and stability in a fast-changing world".
"To those who have been successful tonight - our congratulations," he said.
For 27 years, there has been one party which as been prepared to challenge the establishment.
"Tonight more than ever - that force is still needed."
However, Peters was coy about his next move: "As for the next challenge, we'll all have to wait and see."
Peters and NZ First have held the balance of power three times before.
The party is on about 2.6 per cent, with most preliminary votes counted.
Shane Jones could have offered another way back by capturing the Northland electorate, but he's running third to National's Matt King and Labour's Willow-Jean Prime.
Peters is 75, and the focus will now turn to whether 2020 was his last campaign. Any retirement could end NZ First as a political force, given how closely linked the populist party is to its founding leader.
NZ First held its election night function at the Duke of Marlborough at Russell, in the Bay of Islands, with half the ground floor festooned with black and white balloons.
Earlier, Jones said the results of tonight would be the "fruits of democracy" however the night ended for the party.
But he added that it should be said he and NZ First has stuck to its promise to the regions of the country, pumping investment into areas long neglected.
"No one will ever say of the last three years that NZ First and Shane Jones, for the North and the provinces, did not deliver.
"Northland sadly had been neglected for a long period of time."
Tonight's result is another twist in Peters' political career, which began when he first ran for Parliament aged 30 and standing for National in Northern Maori.
He entered Parliament as a National MP three years later after a court order to overturn the election night result in the seat of Hunua - the first controversy in a tumultuous career.
That has included his split from National in 1993 and the formation of NZ First, being sacked from Cabinet by Jenny Shipley, serving as Foreign Minister under Helen Clark, lean years of support after losing Tauranga in 2008, and upsetting National to capture Northland in a 2015 byelection.
In 2017 NZ First won 7.2 per cent of the vote (nine seats) and the country held its breath as Peters revealed he'd chosen to form a coalition government with Ardern's Labour, serving as Deputy Prime Minister and Foreign Minister.
Peters' pinstriped suits, stump speeches, one-liners and combative approach to the debate chamber and media interviews have become hallmarks of New Zealand's modern political history ("the last of the great characters in an increasingly bland political environment", was how the NZ First website once described him.)
NZ First is defined by its opposition to foreign ownership and a free-market approach to the economy, desire to slash immigration levels, and policies to help its regional and largely elderly constituency, such as the Super Gold card scheme.
The stance on immigration in particular has seen Peters accused of racism by political opponents, including former Green Party co-leader Metiria Turei.
The Covid-19 pandemic buffeted NZ First's 2020 campaign, and took immigration levels out of the spotlight, given the country's closed borders.
Peters attempted to differentiate NZ First from Labour and criticised elements of the Covid response, saying his party would have brought the military in sooner to sort out quarantine facilities, and opened up limited travel with Australia.
He ran as much on what NZ First had stopped (a capital gains tax, for example) than what it had secured (the huge provincial growth fund, for instance).
Peters' campaign started after an undisclosed illness and surgical procedure, and was dogged by a Serious Fraud Office investigation into the NZ First Foundation, an entity set up to handle donations.
Two people are facing charges, it was announced late last month (with name suppression, and not MPs, candidates or current party members) - timing Peters criticised as a "James Comey level error of judgment", with NZ First seeking a High Court declaration the SFO abused its powers.
Peters took a $100 bet with broadcaster Mike Hosking in the lead-in to polling day, saying NZ First would clear the 5 per cent threshold and return to Parliament.
"We have got a surge going on at the moment," he explained. "I can feel it out in the streets and I can see it in the malls."
There wasn't enough momentum, after all, and NZ First faces an uncertain future.