He was speaking with a Silver Fern Farms staff member when he made the quip, which is usually attributed to 19th-century German statesman Otto von Bismarck.
When he was in Dunedin at the start of the election campaign a month ago, Mr Key was under intense pressure from the just-released Dirty Politics bombshell.
Then, he did not look at ease for much of his visit, the strain and discomfort apparent.
Yesterday was different. If he felt pressured by revelations and claims about mass surveillance and the activities of the National Security Agency, it did not show.
Talk at Silver Fern Farms, in its new airy headquarters in the old chief post office, centred on red meat export strategies and the challenges of dealing with foreign markets.
He was relaxed and comfortable chatting with workers about their jobs and life in Dunedin, with little apparent time pressure.
He joked the media pack following him were ''adoring fans''.
Asked pointedly by a worker if the reporters voted for him, Mr Key was good-natured, saying many did not vote at all, as they wished to remain neutral.
After speeches, the floor was opened to questions.
''You can ask me anything,'' he said, after a slow start.
Staff asked about President Barack Obama's golf skills, the price of petrol, the value of the dollar, progress on signing free trade agreements and what he thought of the cooking demonstration put on earlier by Pier 24 chef Michael Coughlin.
''He's a good guy,'' Mr Key said of Mr Obama.
Then followed a series of pointed questions from contractor Vivian Pine, who asked about Dunedin job losses, Dunedin Hospital, Invermay research centre, the closure of Hillside Engineering Workshops and the future of Dunedin's medical school.
''I think there's real opportunity in Dunedin to ramp it up as a producer and an employer,'' Ms Pine said.
Mr Key handled the questions in a relaxed manner with apparent ease.
Economic Development spokesman Steven Joyce also stepped in, disparaging Labour's plan to reopen Hillside.
''With the greatest respect, that stuff's looking fairly back in the past. There's lots of new companies coming now,'' Mr Joyce said.
Mr Key said Dunedin's economy was doing well but its structure was changing.
Ms Pine's questions appeared to surprise some members of Mr Key's entourage.
Mr Key's press secretary, Lesley Hamilton, and Mr Joyce asked the Otago Daily Times who Ms Pine was.
Next up was a visit to technology company ADInstruments, where Mr Key, Mr Joyce, Dunedin list MP Michael Woodhouse and Dunedin South candidate Hamish Walker watched a presentation about the company's projects and goals.
To demonstrate its health IT work, chief executive Michael Macknight projected Mr Key's heart rate on a screen.
''You should ask me some questions I don't like,'' Mr Key quipped.
