Mr Ryall criticised the 81 PHOs, saying there were too many bureaucrats, last year and since then several have merged together.
Four PHOs catered to 30 percent of the population, while at the other end of the scale 41 looked after 12 percent.
This year's budget would have smaller than usual increases for health but costs and the population continue to grow.
PHOs are funded by district health boards to provide essential primary health care services and the Government use them to roll out subsidised services.
Mr Ryall told NZPA this morning he wrote to five DHBs that had not made the progress he wanted.
"I have written to these five DHB chairs to say 'can you start working with these DHBs to get them consolidated'."
Waitemata (six PHOs), Auckland, which covers cental Auckland (five), Northland (six), Capital Coast Health in Wellington (seven) and Hutt Valley (five) were in his sights.
Mr Ryall said some PHOs served only 4000 people while others had 300,000 on their books.
"They've got huge variation in size... The Government is saying that in primary care, as in anywhere else in the health service we need to move resources to the front line and unnecessary bureaucratic structures need to go."
He visited a community where there were four PHOs operating in a relatively small community.
"The job of the immunisation coordinator is now split between the four of them which is hardly a good use of resoucres.
"There's too much duplication, there's a lack of cooperation and we can get a better service for patients if we get some of them joining up. We are spending too much money on administration. We're spending $33 million a year for management fees for PHOs."
Mr Ryall said the PHOs needed money in the budget just to keep subsidies increased to meet inflation and the money needed to be found.
In Otago-Southland nine PHOs were being merged into one and in Eastern Bay of Plenty three had merged into one.
There were talks going on in Auckland and south Auckland.
"It's happening around the country but there are a number of areas where we need to give it a push along.'
Mr Ryall expected progress in those areas.
"I think we'll get further action mid-year."
