Obama, speaking at a memorial in Johannesburg for Mandela, made the comment in front of an audience of leaders that included Chinese Vice President Li Yuanchao, Cuban President Raul Castro and Zimbabwe's president, Robert Mugabe
Earlier South African President Jacob Zuma was booed and jeered at the memorial, a major public humiliation in front of other world leaders six months before national elections.
The hostile reaction from the thousands-strong unruly crowd erupted as Zuma prepared to address the high-profile ceremony in Johannesburg's Soccer City stadium bringing together leaders from around 90 countries.
Zuma, who has ruled Africa's biggest economy since 2009, had been hoping to get a boost from the wave of national emotion triggered by Mandela's death on Thursday, aged 95.
His African National Congress (ANC) government has been facing violent labour unrest and protests over persisting poverty, crime and unemployment. But the ANC is still expected to comfortably win the elections to be held in April or May.
Zuma was booed when he first entered the crowded stadium, and again when he prepared to speak. In contrast, U.S. President Barack Obama received a roaring ovation, and Zimbabwe's Robert Mugabe and U.N. chief Ban Ki-moon were also cheered.
Some in the crowd accompanied the boos for Zuma with thumbs-down gestures and rotating hand movements, the sign for a substitution in a soccer match.
"Mandela had a vision, Mandela lived that vision. But what Zuma speaks, he doesn't live," said Funeka Gingcara-Sithole, 31, reflecting the mood of the Zuma critics in the stadium.
"He should do the honourable thing and resign," she said.
ANC spokesman Jackson Mthembu called the booing a surprise, but played down the incident as a "little blot".
"It came as a bolt to all of us," he told eNCA television. "We were quite surprised."
Zuma and senior ANC officials remained stony faced during the hostile reaction and organisers at one point used a choir to try to drown it out.
The mourning for Mandela has distracted attention from corruption scandals affecting Zuma and his administration.
But memories of the former president's five-year tenure up to 1999 have reminded many just how distant Zuma's South Africa still is from the "Rainbow Nation" ideal of shared prosperity and social peace that Mandela proclaimed after his 1994 election.
