Just under a third of those polled found the comment offensive and racist.
The New Zealand First leader made the joke at the party's campaign launch in Auckland just over two weeks ago.
He was speaking to the audience about how National defended the sale of land to foreign buyers by saying that Labour had done the same when it was in Government.
"As they say in Beijing, two Wongs don't make a right," he said, which brought resounding laughter from the floor.
The joke drew the ire of Chinese group representatives, as well as Race Relations Commissioner Susan Devoy, who called it "outdated rhetoric".
The fallout dominated media coverage of the campaign launch, overshadowing the policy announcement of a KiwiSaver account and a $1000 Government kick-start for every newborn child.
In the survey, 57 per cent of those polled said the joke was a harmless bit of fun that people should not get fussed about.
Just under a third - 32.7 per cent - said it was offensive and racist and just over 10 per cent did not have a view or did not give an answer.
Mr Peters may have simply been playing to his core constituents; broken down into age groups, almost 70 per cent of those aged 65 and over had no issue with the joke and only 22 per cent found it offensive.
Those who were younger, female and Auckland-based were more likely to be offended.
The survey was taken between August 14 and 20. The sample size was 750, with a 3.6 per cent margin of error.
