‘AI’ hype
One of the biggest problems of ‘AI’ is often missed from debates – the fact that it doesn’t work in the ways it is claimed to. So much of the harm caused comes from its lack of functionality.
Arguments around ‘AI’ often focus on an ethical choice – as a 2019 Guardian article on whether it was ethical to use AI in child protection put it: “the other issue for wider debate is whether politicians and the public are comfortable with the harvesting of personal data in this way – even if it does offer the prospect of saving a child’s life.”
“Saving a child’s life” may be the aim but there isn’t evidence that that use of automated decision making systems achieve this, nor that doing so is in any way a prospect. What the evidence conclusively show is that such systems do not work as advertised, cannot work, are fundamentally and fatally flawed and are not appropriate for use in risk prediction.
“Deployed AI systems often do not work. They can be constructed haphazardly, deployed indiscriminately, and promoted deceptively. However, despite this reality scholars, the press, and policymakers pay too little attention to functionality. This leads to technical and policy solutions focused on “ethical” or value-aligned deployments, often skipping over the prior question of whether a given system functions, or provides any benefits at all.”
Drs Inioluwa Deborah Raji and I. Elizabeth Kumar and Aaron Horowitz and Assistant Professor Andrew Selbst
FAccT ’22:The Fallacy of AI Functionality (2022)
“AI hype is playing out today across many products, from toys to cars to chatbots and a lot of things in between….the fact is that some products with AI claims might not even work as advertised in the first place.”
Federal Trade Commission
Keep Your AI Claims in Check
“…systems might seem automated but when we pull away the curtain we see large amounts of low paid labour, everything from crowd work categorising data to the never-ending toil of shuffling Amazon boxes. AI is neither artificial nor intelligent.”
“…with all the hype it can be easy to misunderstand the capability and over-estimate what is possible. Where they have been successfully applied, it is often in a more limited sense than may be apparent from the way it is reported.”
“the onus should be on the company selling the AI tool to proactively justify its validity. Without such evidence, we should treat any risk assessment tool as suspect. And that includes most tools on the market today.”
Professor Arvind Narayanan and Sayash Kapoor
AI Snake Oil: The bait and switch behind AI risk prediction tools (2022)
“AI technology is exceptionally expensive, and to justify those costs, the technology must be able to solve complex problems, which it isn’t designed to do.”
Jim Covello, Head of Global Equity Research, Goldman Sachs
Goldman Sachs, Global Macro Research. Gen AI: Too Much Spend, Too Little Benefit? (2024)
ChatGPT: fancy predictive text
As well as the problem of hype there is the problem that we misunderstand what it is that ‘AI’ is doing. The fact that it is brilliant at some things means we think it can do things it absolutely cannot. For example with tools like ChatGPT the fact that it SOUNDS human makes us think that it is doing human things, that it understands or can provide us with answers. This is dangerous.
“…both lying and hallucinating require some concern with the truth of their statements, whereas [large language models] are simply not designed to accurately represent the way the world is, but rather to give the impression that this is what they’re doing.”
Drs Michael Townsen Hicks, James Humphries and Joe Slater
Ethics and Information Technology: Chat GPT is bullshit (2024)
“The real danger, it would seem, is that humans will simply believe anything the machines say, no matter how wrong.”
Matt Novak
Forbes: Lawyer Uses ChatGPT In Federal Court And It Goes Horribly Wrong (2023)
“…chatbots that we easily confuse with humans are not just cute or unnerving. They sit on a bright line. Obscuring that line and blurring — bullshitting — what’s human and what’s not has the power to unravel society.”
No quick fixes
The problems within machine learning systems – especially of using machine learning for tasks it cannot do – cannot be fixed with small tweaks such as more data or human oversight.
More data
“There are a lot of small data problems that occur in big data. They don’t disappear because you’ve got lots of the stuff. They get worse.”
Professor David J. Spiegelhalter
Quoted in Financial Times. Big data: are we making a big mistake? (2014)
“…racism, sexism and ableism are systemic problems that are baked into our technological systems because they’re baked into society. It would be great if the fix were more data. But more data won’t fix our technological systems if the underlying problem is society….We can’t fix the algorithms by feeding better data in because there isn’t better data.”
“[Allegheny Family Screening Tool] mostly just reports how many public resources families have consumed. Allegheny County has an extraordinary amount of information about the use of public programs. But the county has no access to data about people who do not use public services. Parents accessing private drug treatment, mental health counseling, or financial support are not represented in DHS data. Because variables describing their behavior have not been defined or included in the regression, crucial pieces of the child maltreatment puzzle are omitted from the AFST.”
Human oversight
While some of the worst harms have been inflicted by use of ‘AI’ without human oversight or involvement – such as with RoboDebt where the lack of a human to appeal to left victims with nowhere to turn – that does not mean that human oversight solves all issues and makes it safe. That a system has flagged an issue, or an individual, of course impacts on decision making. Otherwise there would be no justification in paying for it, and indeed we have ample evidence that it does change decision making. Also, a part of the problem is inherent, it is in the way such systems focus on individuals as the problem.
“There is a long-standing body of research shows that, across a wide range of domains, automated decision-support systems tend to alter human decision-making in unexpected and harmful ways.”
Assistant professor Ben Green
Computer Law and Security Review: The Flaws of Policies Requiring Human Oversight of Government Algorithms (2022)