That will include ensuring the public agrees with where it intends to go on this, and that the advantages outweigh the disadvantages.
There are a plethora of considerations, among them whether there are proper safeguards in place to ensure accuracy, avoid damaging errors and protect privacy, that it will not turn out to be more expensive long-term than alternatives and make us beholden to the whims of big technology companies, and how it will cope with those who might find interacting with technology difficult.
It is hard to see how ramming through legislation broadening the ability of the Ministry of Social Development to make decisions using AI gives confidence on these issues.
It is not clear why post-Budget urgency was needed to give MSD the power to ‘‘approve the use of an automated electronic system by a specified person to make any decision, exercise any power, comply with any obligation, or take any other related action under any specified provision, with appropriate safeguards.’’
Not allowing proper scrutiny of such a broad proposal which could affect many beneficiaries is high-handed and short-sighted.
Prime Minister Christopher Luxon likening this move to electronic ticketing in the airline industry was a curious comparison.
He was making the point the change freed staff for other tasks. However, it overlooks that the worst that might happen if electronic ticketing does not work is that you might miss a flight.
While that might have repercussions such as missing a meeting, a family event, or a connecting flight, the hapless passenger is not likely to run the risk of having no income because their benefit entitlement might be wrongly cancelled.
Select committee perusal would have allowed the government to fully explain why this is so important, something it is hard to understand from the documents available.
That process could have been used to reassure the public the government is on the right track, that the extended power will merely reduce the amount of time staff are spending on administration, and that it will only be used where decisions are straightforward with human judgement used where it is needed. It would also allow the adequacy or otherwise of proposed safeguards to be explored.
National’s Scott Simpson promised the new law would allow faster decisions, more consistency, and a system people can trust.
In the highly redacted regulatory impact statement (RIS), officials said it was assumed MSD had the social licence to use Automated Decision Making (ADM). It was assumed the widespread use of automation outside the welfare system might mean the public would support and even expect ADM in the welfare system, the RIS said.
‘‘However, the public may have a range of perspectives which have not been tested.’’
There had been limited consultation on the RIS and the select committee process would ‘‘provide opportunity for wider consultation and input from the public’’.
Except, of course, there was no such opportunity given.
This gung ho attitude by the government has paved the way for critics to draw attention to the misery resulting from the Robodebt scandal in Australia a few years ago where an algorithm was used to see if beneficiaries were getting the right amount of money.
It then spewed out automatic debt notices to people deemed to have been overpaid. If a notice recipient considered this was inaccurate the onus was on them to prove it.
The system was designed to force recipients to respond online and minimise contact with actual public servants, in the interests of economy.
Mr Luxon was quick to say the government had looked at Robodebt and what it was proposing was not comparable.
But has the government heard the clear warning from Catherine Holmes, the head of the Royal Commission into that scandal, about the care required to ensure automated decision-making is wrought for the public good?
‘‘The stakes are high: every serious misadventure in the implementation of automated decision-making processes will diminish the credit society extends to government.’’
