Les Bleus brought the Stade de France to the boil after a timid first half, turning on the style when it mattered to make a giant leap towards securing top spot in Pool A.
They scored two tries through Damian Penaud and Melvyn Jaminet, while the boot of fullback Thomas Ramos made the difference with 17 points.
The All Blacks, beaten for the first time in 32 World Cup pool games, had two Mark Telea tries and a sole penalty by Richie Mo'unga to show for themselves.
France next face Uruguay in Lille on Thursday while New Zealand will take on Namibia in Toulouse the next day.
"It was a close game but the most important was to win tonight. Thanks to the crowd, who were outstanding," said winger Penaud.
While Les Bleus seemed to be on the back foot with very few chances early on, they were just lying in wait in the Parisian humidity, leaving their opponents wasting energy with repeated runs in the suffocating heat.
Fabien Galthie's side abandoned possession in the opening half before turning on the engine after the break, finally showing the kind of rugby that has put them back on the map in the last four years.
Sixteen years after losing their opening World Cup game as hosts against Argentina, France rose to the occasion, beating the All Blacks for the second time in a row.
The result leaves Ian Foster's team with a lot of questions to answer and doubts to shake off following their record loss against South Africa last month.
