No consent
Many of us are unaware of how our data is being used, that automated decision making systems are used (‘AI’) or why it matters. It isn’t something we think about. But when we understand the huge impacts it has we do care. Very much.
Government and local authorities have gone far beyond their mandate in their use of children and families information. There has been no discussion about whether citizens consent to this type of governance. Information is used without informed consent. Algorithmic automated decision making systems are used to profile people in ways we know to be inappropriate and which are evidenced as leading to negative outcomes.
Children and their families are being hurt by this and they didn’t consent.
“…there had been very little consultation with citizens about the implementation of these systems and also in many cases not really a proper kind of discussion around informed consent or the fact that their data was being used for these types of purposes. And that matters.”
Dr Lina Dencik
The Alan Turing Institute: Social Justice in an Age of Datafication (2019)
AI: no mandate
The public do not want ‘AI’ to be rolled out in the way the government is planning to do. The government knows this, their own research tells them so, but it has been used across the public sector and they are rolling it out much more widely now.
Efforts are focused on making the public feel reassured about safety standards within AI and are avoiding the bigger question of whether AI should be used in specific areas at all.
do we even want algorithms making significant decisions?!
“..we call for Government to pay heed to the evidence on public trust and press pause on the data sharing and Artificial Intelligence (AI) systems for public services”
Professors Rosalind Edwards and Val Gillies and Dr Sarah Gorin
Evidence submitted to House of Commons Science and Technology Committee (2022)
“…similar to the use of AI in healthcare, [AI to design lesson plans] feels more controversial and participants get naturally hung up on the debate about whether it should be introduced at all.”
Thinks Insight and Strategy
(Report prepared for Responsible Technology Adoption Unit, UK Govt)
Public Attitudes to AI Assurance (2024)
“As understanding and experience of AI grows, the public is becoming more pessimistic about its impact on society. A growing proportion of the public think that AI will have a net negative impact on society, with words such as ‘scary’, ‘worry’ and ‘unsure’ commonly used to express feelings associated with it.”
Department for Science, Innovation and Technology, UK Government
Public attitudes to data and AI: Tracker survey (Wave 3) (2024)
“…in some cases participants do not agree with certain uses of AI technology. This means they automatically view these applications with mistrust and therefore find assurance less reassuring. This is often due to the risk they pose to job loss or human connection…”
Thinks Insight and Strategy
(Report prepared for Responsible Technology Adoption Unit, UK Govt)
Public Attitudes to AI Assurance (2024)
“Only 10% of [social workers asked] thought that predictive analytics had a role to play in decision making in social care”
What Works for Children’s Services
Machine Learning in Children’s Services summary report (2020)
AI: no transparency
Far from being transparent and open about the use of algorithms evidence shows cover up and secrecy. Researchers found that “local authorities are not transparent about their use of operational data linking and matching and the application of predictive analytics, providing incomplete, insufficient, irrelevant and ambiguous information even when it is clear they are using data analytics.”
The UK Government has been secretive about their use of algorithms. The majority of people whose information is being shared and monitored, and assessed by inappropriate automated decision making systems have no idea. Not only are they not asked for consent but evidence is hidden from them.
The Public Law Project developed a tool to track automated decision making used in government and found that only one out of 55 automated systems could be described as highly transparent whereas 38 are classed as low including the Troubled/Supporting Families programme. Nearly all of the uses were only uncovered or more fully understood via an investigation and use of Freedom of Information requests. Half don’t even have publicly accessible assessments on their impact on the protected characteristics of individuals.
Not only this but the government has gone to court to NOT admit that they are doing this. When campaigners noticed that disabled people seemed to being investigated at a disproportionate level – despite fraud in disability benefit claims being tiny, 0.5% – the government refused to answer any of the Freedom of Information requests of the Greater Manchester Coalition of Disabled People and of Foxglove.
“…how can government maintain public trust in services and the use of public data, when councils are applying undisclosed algorithms to public data without our knowledge or consent, for unevaluated interventions, to screen their populations for a problem as serious as child maltreatment?”
Professor Ruth Gilbert and Rachel Pearson and Professor Gene Feder
Guardian Letters: Don’t trust algorithms to predict child-abuse risk (2018)
“It is important to note that people’s awareness of technologies for assessing risk and eligibility is relatively low. Some of these technologies are already being used in public services, and these results show that people may be largely unaware of the technologies that help make decisions which directly impact their lives.”
Alan Turing Institute and Ada Lovelace Institute
How do people feel about AI? A nationally representative survey of public attitudes to artificial intelligence in Britain (2023)
“..we recommended that the government should place a mandatory transparency obligation on all public sector organisations using algorithms when making significant decisions affecting individuals. This would require the proactive publication of information about the algorithms.”
“Our research points to the lack of any systematic information about where and how data systems are being deployed across government.”
Professor Lina Dencik, Dr Arne Hintz, Professor Joanna Redden & Harry Warne
Data Justice Lab: Data Scores as Governance: Investigating uses of citizen scoring in public services (2018)
Data sharing: no mandate
While some data sharing is necessary and useful there is no mandate for ‘frictionless’ data sharing. We know that children, families and the wider public care about how their data is used, the impact it can have on them and want to keep control of it. This is not the reality. Instead all the power lies in the hands of the state and private corporations.
Moreover we do not trust the government to use our data in a way that is good for us. Research by the government showed that the only type of organisations we trust less are social media companies. When surveyed about trust in various areas the government are significantly less trusted than even the big tech companies – and we do not trust them. There is low trust in government; to effectively use data to improve services; to keep data about us safe; to use our data to benefit society; to be transparent and open about what they do with our data or to let us make decisions about what is done with our data.
This is unsurprising given a track record of secrecy across government after government. Repeatedly harmful legislation has been introduced with little public discussion.
The research shows that we still trust the NHS, but this might well be damaged by examples of misuse of our data.
“UK adults express a clear preference that they are not personally identifiable in data that is shared.”
Department for Science, Innovation and Technology, UK Government
Public attitudes to data and AI: Tracker survey (Wave 2) (2022)
“Families desire to keep control of their data was often motivated by fear; they expressed a desire to keep control when they perceived risks of it being shared or misused, fearing the consequences of mishandling. Often implicit in this was a distrust of authorities.”
“NHS trusts are sharing intimate details about patients’ medical conditions, appointments and treatments with Facebook without consent and despite promising never to do so.”
“The NHS’s failed care.data project, which closed following widespread public distrust of the data collection and sharing it required, showed that if these systems are to work they must reflect what families and citizens want and need.”
How do children feel?
When children are asked they do not like information about themselves to be shared without permission and have concerns about their privacy and the harm information shared can cause them. When surveyed on information on their education and schooling being shared over 90% stated that they did not agree with categories such as attendance, behaviour, families finances, child health or special needs or educational attainment being shared. 67% felt that none of the information should be shared at all.
“Our research has found that education data are widely shared with commercial companies, but it has proved near impossible to discover which data are shared with whom for which purpose. Our survey results found that children generally think such sharing is unacceptable. They were especially concerned about sharing more private information such as health or financial data.”
Professor Sonia Livingstone and Dr Kruakae Pothong
Digital Futures Commission: What do children think of EdTech or know of its data sharing? Read our survey findings (2022)
Children were asked how they felt about ContactPoint, the controversial national register of children of 2009. Children had quite a lot of concerns. We need to listen to these. The current proposals to implement a ‘unique identifier’ would mean huge amounts more sharing for every single child in England.
“u can’t put too much trust in a machine”; “letting the children know what information is going to be held on them before its up and running”; “to get our permission before people get access to our information”; “to inform us of who is using the information and tell us what is being put on there.”
Children consulted on ContactPoint quoted
Roger Morgan, Children’s Rights Director for England
OFSTED: Making ContactPoint Work Children’s views on the government guidance (2007)
[Children in our discussion] “thought that paedophiles would spend a lot of time and effort trying to break into ContactPoint. They thought that one day, eventually, the system would either break down, or its security would be breached, and that would have very bad consequences for children and, they thought, for the Government.”
Roger Morgan, Children’s Rights Director for England
OFSTED: Making ContactPoint Work Children’s views on the government guidance (2007)
Data linkage
Many people do not understand – let alone consent to – how much data can be linked, and what that means, or the changing nature of data retention
“Prior to 2012, bringing together identifiable data profiles on this scale would have been categorically unlawful, but while arguments were raging as to whether troubled families existed and how to work with them, new legislation was quietly introduced to provide de facto exceptions, allowing data to be linked without families’ consent, for the specific purpose of identifying and monitoring “troubled families”
Professors Val Gillies and Ros Edwards and Dr Helene Vannier Ducasse
De Gruyter Oldenbourg : Calibrating families: Data behaviourism and the new algorithmic logic (2024)
Did you know that the retention period listed for the Department of Education to use child and parent information is “your learning and working life” – until you are 80, or retirement plus three years. Linked information about numerous parents is collated as part of the Pupil Parent Matched database, which is linked to the National Pupil Database. Currently this database is not shared at named level and is for research only but there is no guarantee that this will remain the case, especially should it be considered information about ‘safeguarding’. The information held about children in the National Pupil Database is already shared, including with the Home Office and with private corporations, and there is no clear reason why information about parents will be viewed differently.
how data linkage can be used
This demonstration by firm Neo4J – who are a government supplier – shows how their software can be used by police or for child protection and the huge amount of information and inferences that can be drawn from it. It is a long video – 52 minutes – but eye opening for those who have time.
how family information is used now
A research project, Parental Societal License for Data Linkage for Service Intervention, shows how difficult life events and data sharing can result in child protection assessment. This short video aims to help parents understand how information is used, and the possible risks.
Case study: Somerset Council
Collecting and sharing large amounts of information about residents is justified as ‘improving outcomes’ yet in reality struggling councils, battered by austerity, have less and less to offer residents. It isn’t helping children or their families.
The council’s privacy notice explains that the “Digital Economy Act (2017) allows the Council to share information with professionals (Trusted Partners) for the purpose of improving outcomes of children, young people, and their families”.
Information about a family will be visible on their data sharing platform Transform for a wide variety of reasons including if: child has any special educational needs or disabilities, or needs any additional support; a child or family member has any health needs; your family has any financial difficulty, are out of work or experience debt or rent arrears; your child experiences barriers attending school. Other organisations, for example, Young Somerset which provides activities for children and youth work support, collect and share the information about every child and young person who accesses their services (not only if there is a safeguarding concern) and high level information is added into Transform.
Information is held on Transform anywhere from 6 years from case closed in cases of contact and referral only, until a child is 25 years if any early help or a child in need assessment is carried out, 35 years for ‘Early Support’ for children with disabilities, 35 years if a child protection assessment (s47) with NO child protection outcome is carried out, 100 years from child’s DOB if child protection plan put in place or any further measures taken.
“It is clear we are one of the growing number of councils nationally to be facing a financial emergency. There is a fundamental structural problem with local government budgets as our costs are rising much faster than our ability to raise income.”
Cllr Liz Leyshon, Deputy Leader
Somerset Council website
“The austerity agenda as a force leading local authorities to turn to data systems for help was stressed by interviewees across local authorities and is also referenced as a driving force in government documents and corporate promotional material.”
Professors Joanna Redden and Lina Dencik, and Harry Warne
Policy Studies: Datafied child welfare services: unpacking politics, economics and power (2020)