Prisoners will have a year to adjust to not being able to smoke in jail but after that new inmates will have to go cold turkey.
Corrections Minister Judith Collins said the ban, which she expected to take effect from July 1 next year, would make prisons healthier for staff and inmates.
At the moment people can smoke in their cells and some outdoor areas.
"The high level of smoking in our prisons poses a serious health risk to staff and prisoners. Studies of air quality in United States prisons show that staff and prisoners can be exposed to 12 times the levels of second hand smoke than in the home of an indoor smoker," Ms Collins said.
The Corrections Department was concerned that staff or prisoners might sue if their health was harmed.
"Two thirds of prisoners are smokers and the most common health risk factor reported among prisoners is tobacco smoking. Prisoners have triple the rate of smoking as the rest of the community," Ms Collins said.
Prisoners would have access to the Health Ministry quit programme that includes nicotine replacements.
Corrections Association president Beven Hanlon told Radio New Zealand today that while the union supported the policy there could be tensions with addicted prisoners.
"People coming off nicotine can be very unpredictable, can be very anxious, aggressive and we're going to have a large part of our prison population going through that and we're (prison officers) going to have to manage them," he said.
Rethinking Crime and Punishment director Kim Workman said it would be particularly difficult for new prisoners who were already grappling with drug and alcohol withdrawal, and mental and physical health issues.
He said the move was likely to cause "violence or mayhem of some kind".
A black market in tobacco was also likely, he said.
Prime Minister John Key said he supported banning smoking in prisons which had been done successfully in Australia.
There would need to be programmes to help prisoners quit, he said.
"I think it takes a period of time. I think it's quite cold turkey actually... eventually there'll be a day that they say that's it, they're smokefree and we'll go from there."
