They were eight points shy of India who took the crown - and a $US1 million purse - for the third consecutive year.
The International Cricket Council's world test championship begins in August, with nine nations taking part in 71 matches spread across 27 series to reach a 2021 final.
"It's an awesome achievement for our side to finish second in the ICC test team rankings and as a team we should be very proud. It's come off the back of a lot of hard work and perseverance; not just by the eleven guys on the field, but the entire squad and support staff," Black Caps skipper Kane Williamson said.
"It's been especially pleasing to see the consistency we've shown in the longest form of the game over a decent period of time. We're now all really looking forward to the ICC World Test Championship getting underway and the added context the competition will bring."
Test rankings
1. India 116
2. New Zealand 108
3. South Africa 105
4. Australia 104
5. England 104
6. Sri Lanka 93
7. Pakistan 88
8. West Indies 77
9. Bangladesh 69
10. Zimbabwe 13
