The 33-year old shooting guard racked up 29 points and six assists against the 2009 European Championship runners-up, whose coach Aleksandar Djordjevic expelled forward Vladimir Micov from the national team for dissent.
The much-travelled Penney, whose myriad of clubs included brief spells at the Miami Heat and the Los Angeles Clippers, rolled back the years and nailed five of his seven three-point attempts in the Pionir Hall.
"I can't blame my players for lack of effort and I must also give credit to New Zealand because they hit a barrage of difficult shots inspired by the unstoppable Penney," Djordjevic told a news conference.
"As for Micov, his short national team career is finished. He lashed out at the staff when he was benched and that's unacceptable. I wanted to have words with him but he just turned around and showed disrespect."
The incident was another blow to Djordjevic's injury-hit squad, aiming to better their fourth-place finish at the 2010 tournament while rekindling past glory when Serbia won a haul of medals within the former Yugoslavia.
But New Zealand, who drew a standing ovation from the Belgrade crowd when they performed the haka before the tip-off, overcame an early nine-point deficit on the back of Penny's 20 points.
Tightly marked in the second half, Penny still delivered crunch shots and also dished out defence-splitting passes to his team mates as the Kiwis silenced the home fans.
Serbia won the first match between the pair.
New Zealand's preliminary pool rivals at the Aug. 30-Sept. 14 tournament include the United States, while Serbia were pitted alongside European champions France and hosts Spain.
