The Southlander claimed 5-34 to help bundle out West Indies for 167 in a spectacular collapse from 100-2 at one stage at Christchurch's Hagley Oval today.
Opener Tagenarine Chanderpaul and Shai Hope made half-centuries in their otherwise feeble reply to New Zealand's first innings total of 231.
The Kiwis finished a stop-start day two on 32 for no loss, for an overall lead of 96 runs.
Tom Latham (14) and Devon Conway (15) will come out tomorrow hoping to bat West Indies out of the contest.
Earlier, New Zealand took two early wickets when the track looked lively but Hope and Chanderpaul put on 90 for the third wicket.
Hope fell caught behind off a Duffy short ball for 56 and Matt Henry (3-43) drove home the advantage by removing touring captain Roston Chase and Justin Greaves in the space of four deliveries.
Chanderpaul did not look convincing at the start and was dropped twice but he dug in, found his rhythm, and raised a hard-earned 50.
His marathon 169-ball stay finally ended on a pull off Zak Foulkes that did not get past square.
An airborne Devon Conway took a stunning catch to make amends for an earlier drop.
When play resumed after a rain break, Duffy sent down a two-wicket maiden over to rattle West Indies and removed Jayden Seales and Ojay Shields to hasten their collapse.
"Obviously they got a bit of a partnership going, we knew they would at some stage, and it coincided with the ball stopping swinging," all-rounder Foulkes said afterwards.
"Duff has taken on that enforcer role, which is awesome. It's great to see. And then we actually saw it swing again at the end, which was interesting."
