Go to top of page

How Centrelink unleashed a weapon of math destruction

7 January 2017
Peter Martin
SMH

Unveiling the automated Centrelink debt recovery system mid-year treasurer Scott Morrison and social services minister Christian Porter promised more "accurate and appropriate income testing". They were going to work with the prime minister's Digital Transformation Office to "cut red tape and ensure that mistakes are minimised".

[...]

What Morrison and Porter promised was an automated system that would issue Centrelink debt notices "better" than human beings.

Humans did the job extremely well. A former Centrelink worker with 30 years experience says they would "look at start dates for employment that customers had declared, see if it was the same for the employer [using Tax Office records] and roughly work out if it lined up."

[...]

What's important in this description is the humans charged with applying the law didn't issue debt notices unless they had evidence that a debt existed. To do so without evidence would be to break the law. But a wrongly-programmed computer need have no such scruples. Even better, its decisions can be presented as objective, hard to overturn.