He believes it's because of comments he made about author Eleanor Catton during the "tall poppy" debate and the supposed intolerance of New Zealanders to dissent and intellectual debate.
Plunket said he had always been happy to be involved with the festival and when the organiser rang him a couple of months ago to ask if he would like to be involved with chairing sessions again, he said yes.
However, his invitation has now been withdrawn, with the organisers saying there would be too much emphasis on him rather than the writers who would be appearing.
Plunket drew criticism in January after he branded Catton an "ungrateful hua".
His comments came after Catton told an audience at the Jaipur Literary Festival in India that she struggled with being an ambassador for New Zealand because she did not think the country was doing as much as it could to support the literary arts.
Punket said on his radio show that he did not consider Catton an ambassador for New Zealand but "a traitor", and called her an "ungrateful hua".
