The claim comes from Tertiary Education Union (TEU) national president Sandra Grey.
And the TEU also said that if the university's governing council was insufficiently representative, this could also indirectly harm Dunedin's wider interests.
The four people reappointed this week were chancellor John Ward, and council members Stephen Higgs, Dr Royden Somerville, and Donna Matahaere-Atariki.
Mr Joyce said there were ''a number of balancing criteria'' he had to consider in making such tertiary appointments, he said.
He noted that all four Otago appointees were reappointments.
''And I can't recall any criticism by the TEU of any of the four people in the past,'' he said.
Ms Grey said the recent amendments to the Education Act required Mr Joyce to reflect the fact - as far as was ''reasonably practicable'' - when making appointments to tertiary institutions, that about half New Zealand's population was female.
His latest Otago appointments - to a council comprising two-thirds men - involved reappointing three men and one woman.
The minister was also required by the same law
to reflect the community's overall social and economic diversity.
But his Otago appointments comprised a chartered accountant, a company director, a barrister and a management consultant.
Since the February law changes, Mr Joyce had made 35 appointments or reappointments, of which 22 were men.
Only two of those 35 were not company directors, managers, partners, consultants, commercial lawyers or accountants.
