Mr Key told TV3's Paul Henry Show that he didn't like the policy around the detention centre.
"The whole way through we've said we don't like New Zealanders being sent there... Our point is we don't like that, we don't like what's happening."
He said New Zealanders on the island can come back to the country.
"You can come back to NZ and you can come back in a few days, if you have no issues, or potentially a few weeks. So people are electing to stay there.
"Firstly, they may not have a passport because it may have been cancelled a while ago. Secondly, they may have mental health issues or a history of violence, we can't put them on a commercial plane."
New Zealanders on Christmas Island can return home to New Zealand, but were fighting the deportation orders, Mr Key said.
"They are absolutely free to come on home on a plane, they don't need to be on Christmas Island and that can happen within days."
Mr Key said he didn't have the details of what crimes the New Zealanders on Christmas Island have committed.
He said it wasn't true that some of the Kiwis detained had committed only minor crimes and he stood by his comments made in the House yesterday.
"A third of all of the people that actually are in the detention centres... that are New Zealanders, are either rapists, they're murderers, they're child sex offenders or they are responsible for a serious assault," he said.
Meg de Ronde, Amnesty International's New Zealand campaign manager, told TV3's Paul Henry Show New Zealand should have spoken out on Australia's detention centres at the UN conference in geneva.
"Around 75 per cent of countries that spoke raised these concerns around how they treat refugees and asylum seekers, specifically around offshore detention centres and detention," she said.
"This is the chance to get it on the record.... We know that pressure from international community is incredibly important."
