Academics query software use for risk assessment

James Maclaurin
James Maclaurin
University of Otago researchers are warning of potential problems in government departments using computer-based risk-prediction models, including a controversial new tool used by ACC to profile clients.

Associate Prof James Maclaurin, spokesman for the university’s Artificial Intelligence and Law in New Zealand Project, said the ACC used a computer model to assist staff managing claims.

More details about how the system operated should be provided, to ensure it was used in a "fair and even-handed way", and was providing "good outcomes", he said.

ACC had recently indicated the tool was being used to make several kinds of predictions, including how long ACC would expect a claim to be managed. This description left open the possibility that ACC used these predictions to minimise treatment times, either by intervening in patients’ treatment, or "more seriously" by declining applicants with long predicted treatment times, he said.

The tool makes predictions about future ACC cases using a database of information about 364,000 past claims  lodged between 2007 and 2013. Fellow project researcher Prof Alistair Knott, of the Otago computer science department,  said this kind of predictive system guided a corporation whose decisions had "serious consequences for people’s lives".

But it was essential that governments and companies relying on them could answer several questions, including how accurate the tool was and how it had been evaluated.

The researchers also asked that if by "passing the buck" to the machine, ACC was also ducking its responsibility to make "fair and humane decisions" about treating people in need. They also asked if the tool implicitly discriminated against individuals on grounds such as age, ethnicity or gender. ACC’s  use of "data and analytics" told staff  about how people were injured,  their  likely recovery time and how to help.

Since ACC started using its predictive tool, client satisfaction rose from 68% in June 2013 to 78% today, and the average time to set up weekly payments had dropped from 11 days in June 2014 to seven days, ACC said.

john.gibb@odt.co.nz

Add a Comment