The agency's director of living well Martin Hefford was speaking at the Building the Future of Health hui this week, a regular webinar to update health professionals - and anyone else interested - on Te Whatu's work.
Hefford said the "medical cavalry" was not coming to get the country out of the shortage of general practitioners, which was predicted to grow to 1000 by 2033.
About 40 to 50 percent of GP work could be done by telehealth consultations, he said.
Telehealth is when patients have appointments with their GP via video or phone calls.
Because the location of GPs did not always align with where there was the greatest need, telehealth could give more people access to care, he said.
And help could come from outside of the country too.
"It's a great opportunity to balance demand in rural areas, demand in areas with high need populations, with people who might be operating from anywhere in New Zealand or anywhere actually in the world, providing they are New Zealand trained or New Zealand accredited," he said.
One major GP group said it was "dismayed" by the comments.
The General Practitioners Owners Association chair Angus Chambers said it showed Te Whatu Ora did not understand the work of GPs.
Telehealth was a great complement to general practice for matters that did not need face to face intervention, but there was other care that could only be achieved by being in the same room with someone, he said.
That included "opportunistic care", he said.
That was where doctors might notice another problem or take the chance to talk to their patients about a matter they had not come to the doctor about.
He was also annoyed by Hefford's suggestion that non-doctor roles play a greater part in clinical care - such as nurses, health care assistants, physios and health coaches.
"This is yet another example of Te Whatu Ora misunderstanding and undervaluing GPs' work. Mr Hefford also claimed that 20 percent of general practice work could be undertaken by physiotherapists. This is just ludicrous and assumes that the diagnosis has somehow been achieved without a GP," he said.
Hefford said there were 21 million "GP engagements" in New Zealand last year.
About 5 million New Zealanders, or 96 percent of the population were enrolled with a clinic.
But for the rest there were still barriers to care, such as being able to get an appointment or not being able to afford to go.
Work had to be done to break those barriers down, and find new ways to make sure everyone got the care they needed as doctor numbers shrank.
However, Health New Zealand said greater use of telehealth services would "free up" GPs to focus on complex patients.
The National Clinical Director of Primary Care, Dr Sarah Clarke, said the agency was working hard to boost the GP workforce with extra funding and training places - but there critical shortages remained.
"Despite our best efforts, we still don't have enough GPs, and we will have to work together as a system to redirect some of the tasks GPs do so they can focus their time on the complex care that is done best by our valued GP workforce," she said in a written response to RNZ.
"We know that continuity of care in general practice is important, and that being with the same provider for at least 15 years significantly reduces the risk of death, the need for acute care after hours, and hospital admissions. In a model where you know your patient, continuity can be maintained while a lot of care is provided via telehealth, as is already happening right across Aotearoa."
Many practices were already providing a large number of services via telehealth, such as letting patients check test results or request prescriptions online, she said.
"We use telehealth every day, as it is one way to provide quality care that is efficient and convenient for people, their whānau and for communities."
Te Whatu Ora defined "telehealth" in this broader sense, not simply video or telephone consultations, she said.
