While there has been some controversy over the government’s reliance on big data to save taxpayers’ money, none of the commentary has come close to understanding the complexities hidden in the notion of the error rate.
The system that Centrelink employs is an example of artificial intelligence, and the problems it faces are intrinsic to all decision systems.
The true positive rate is the percentage of debtors sent a letter, and the false positive rate is the percentage of people incorrectly sent a debt letter. This graph [see original article] is generated by marking all true positive/false positive pairs for each possible decision line against the same simulated data used earlier.
There are two things we can see from this graph: if you want to identify all of the debtors (100% true positive) you will also catch 90% of the debt-free in your net. This point is marked A on the curve. Furthermore, if you do not want to write any unnecessary letters (0% false positive) then you will only find 38% of those who should pay back a debt (marked B).
As usual, a trade off is unavoidable, and the sensible way lies somewhere in between. That’s why you have to accept that there will always be some errors in your system. The challenge is how to balance them, and how to deal with them whey they are occur.
One puzzle remains for the Centrelink system. The data matching algorithm has been in use for several years in a manual form, but only now is it making headlines.
The answer lies in the expansion of its application. The automated system sends out 20,000 letters a week, a 50-fold increase on the the manual system. Broadening the population to which a system is applied has diminished its effectiveness.
The Human Services Minister, Alan Tudge, says that sending a letter to a non-debtor is not an error.
But, as we have seen, downplaying these errors and concentrating only on identifying more and more debtors magnifies the number of this second type of error.
Also, expanding the data-matching system and removing the human element from the case-selection process has undermined the system’s performance even further.
By underestimating both the number of errors possible and the effects of their interaction, Centrelink is left dusting itself off after having been hit by a big steaming pile of data.