But he then went on to say he believed it could be done.
It was easy to spend time focusing on a fourth term, he said in brief opening remarks to the party's 80th annual conference in Christchurch.
"I don't think we should focus so much about that factor.
"The voters of New Zealand don't concentrate so much on which term of Government you are in but about the results you can achieve for New Zealanders, the focus of attention you really have on the issues that really matter - the economy, law and order, health and education - what you are going to do for the voters and what your vision is for New Zealand."
The Government had to earn the right to govern every day.
It was judged on on every decision it made, good or bad.
"We get judged on the way we handle the decisions, not on the complexity of them, certainly not by whingeing about how difficult they might be but by actually coming up with solutions then delivering on them."
He said National had delivered stable and predictable Government and it had to remain committed to the task.
"There is absolutely no room in my view in the Government for complacency, for arrogance or for mediocrity.
"If we keep delivering every single day, if we go into that campaign in 2017 energised with new policies and new approaches and new ways of tackling the issues that our country faces, I am absolutely confident we can continue to deliver the electoral success we have in the past.
"We have to earn the right to do that but this party is capable of doing that."
Mr Key paid tribute to deputy leader and Finance Minister Bill English, the two of whom have been a team for 10 years.
"I think any successful Government needs to have a leader and a deputy leader that are attached at the hip."
- By Audrey Young
