Simon reunited the original Graceland band led by Soweto guitarist Ray Phiri for the Hard Rock Calling festival and brought along Ladysmith Black Mambazo, the Zulu acapella group whose original rendering of the song "Homeless" shot them to worldwide fame.
A host of other stars helped out, including Hugh Masekela, the legendary South African jazz trumpeter who was exiled under apartheid, and reggae king Jimmy Cliff.
As the crowd in London's Hyde Park swayed and danced to the chiming guitars, punchy horns and accordians of the township jive, it was hard to imagine such joyous music had ever caused a political storm.
Back in 1985, Nelson Mandela was still in jail and the apartheid system of white minority rule held South Africa's black majority in a vicious grip.
Inspired by a tape of township music, Simon flew from New York to Johannesburg to record tracks with black musicians. But in doing so, he violated a U.N. cultural boycott.
When Graceland was released in 1986, it drew the wrath of many anti-apartheid activists, including the African National Congress. Critics accused Simon of exploiting the musicians and boosting the apartheid government.
But the album sold millions, alerted Western rock fans to South Africa's music and its problems, and revived his flagging career. He eventually made his peace with the ANC and South Africa became a free democratic nation with Mandela's election as president in 1994.
The Hyde Park show opened with a run of songs from Simon's career which served as a reminder that he was a pioneer in embracing music from across the world - the Hispanic rythmns of "Me and Julio Down by the Schoolyard", the zydeco stomp of Graceland's "That Was Your Mother", samba drums in "The Obvious Child" and the reggae-influenced "Mother and Child Reunion", which he sang with Cliff after the Jamaican had delivered a brace of his own hits.
Ladysmith Black Mambazo came on stage to huge cheers, their sonorous harmonies resonating across the park. Simon joined them for "Homeless" then launched into the Graceland songs.
Many of the crowd, a sizeable portion of whom were young, appeared to be word perfect, singing along to "I Know What I Know" and filling in the horn parts on "Diamonds on the Soles of her Shoes."
The show reached an emotional point when Masekela came on. "We negotiated the way to freedom in South Africa and now we ask you to shake some booty for a man who is going to be 94 next week - Nelson Mandela," he said.
He then played his song "Free Nelson Mandela", an anthem that was banned in South Africa at the time.
Simon, now aged 70 and looking somewhat dumpy in a black t-shirt, blue jacket and pork pie hat, at first glance cut an unlikely figure amid the African personalities on stage. But their affection and respect for him was clear and his musical skills were evidently supreme.
The Graceland show wrapped up with romping versions of the title song and the hit "You Can Call Me Al". He returned to play two old Simon and Garfunkel hits, "The Sound of Silence" and "The Boxer", harking back to the early, folky days of his own musical journey.
Among the crowd were Amy Clewes, a London theatre administrator, and her boyfriend Sam Garrett. Both were born in the year of Graceland's release but were nonetheless in its thrall. She had bought the tickets as his birthday present. "It's appeal? It just makes you happy," she said.
Garrett said his parents used to play it in the car on family holidays. He was vaguely aware of the original controversy, but, he said: "Its musical creativity far out weighs its political impact."
