The Dairying and Clean Streams Accord 2008/09 snapshot report released told "a totally unacceptable story of effluent management", he said.
"Regardless of whether this is because farmers don't have the right tools, don't know how to comply, or simply don't care, behaviour has to change," Mr Carter said.
The dairy industry overall would suffer from damage to its reputation unless it could back up its claims of sustainable dairying with action and evidence.
"You can argue the merits of dairy to our economy until the cows come home - but until every farmer takes responsibility for improving effluent management, the environment and dairying's reputation will suffer," he said.
"I am putting non-complying dairy farmers across the country on notice. You need to take individual responsibility for this issue and work more effectively with your neighbours, your regional councils and your industry body."
Regional councils and other partners also had to lift their game, he said.
"They need to work harder at identifying those farmers who want to comply but need some help, and support them toward compliance.
"Regional councils also need to be more consistent in their monitoring, and be more rigorous in their application of existing regulations to penalise those flouting the law."
Regulation was a possibility, Mr Carter said.
"Once we have supported those farmers who want to comply, we can look at whether existing regulation needs to be strengthened to target those remaining farmers who blatantly pollute."
On the bright side industry giant Fonterra had ensured 99% of dairy farmers have a tailored nutrient budget, Mr Carter said.
"But the dairy industry needs to take the next step and see effective nutrient management systems in place, supporting farm efficiency and minimising environmental impact.
"Fonterra has assured me it will introduce measures to meet this challenge and I welcome this response.
"Unfortunately, the figures for 2009/10 are unlikely to bring us any closer to our vision of sustainable dairying in New Zealand, as the reporting year is almost over. But this time next year, I expect a report back on action taken to tell a more positive story."