iscover
Share this article on:
Powered by
MOMENTUM
MEDIA
Over 275 automated decision-making (ADM) tools, including AI, have been uncovered being used by NSW government entities.
A survey of NSW public sector entities has revealed a list of ADM tools that are either in use or being developed for use within three years, with technologies including artificial intelligence (AI) chatbot ChatGPT.
The survey uncovered 275 ADM tools, which is likely a conservative number as only roughly a quarter of the 439 NSW public sector entities participated.
Furthermore, an additional 702 potential ADM tools were discovered following a public review of additional sources, including government websites, procurement records and other publicly available information.
The discovery of so many automatic processes within the state’s public sector has ignited concerns about how ADM tools are used and the circumstances in which their use is necessary.
“We believe that members of the public whose rights and interests have been materially affected by a decision made with the use of ADM are entitled to be informed of the role ADM played in that decision,” said NSW Ombudsman Paul Miller in a report titled A map of automated decision-making in the NSW Public Sector.
“Visibility is necessary for people to properly consider and exercise any decision review rights as well as for proper oversight.
“It is also key to supporting an informed debate about what assurance and regulatory frameworks may be appropriate for ADM use now and into the future.”
In his new report, which aims to increase the visibility of ADM use within the NSW public sector, Miller has called for mandatory ADM reporting.
“We do not know in respect of each system how it was designed, legally validated, or tested, and whether and how it is subject to ongoing monitoring for issues such as accuracy and bias,” he added.
“Of particular concern to us, it was reported that less than half of the ADM systems reported to the research team had any legal input at the design stage.”
The report believes there will be a 50 per cent increase in ADM adoption by government entities over the next three years.
The report also distinguished between “fully automated” and “partially automated” ADM tools, with the former making decisions and the latter advising them.
Miller said that even if a human being plays a role in the decision-making process, the ADM still influences the rights of humans.
“An important aspect to resolving definitional challenges was recognising that ADM includes systems that support decision-making in addition to systems that fully automate decision-making processes with minimal or no human intervention,” the report said.
“This is important because agencies sometimes erroneously assume that if a human is present at some points in a decision-making process, then the system is not an ADM system and the issues relevant to ADM use, including those we explored in our 2021 report, are not relevant.”
Miller’s report calls for legal consultation and regulation. A second survey conducted for the report found that out of 26 respondents, only nine said the ADM design team included legal experts, while two said they were unsure.
In contrast, when questioned about the authorisations ADM tools were given, 10 said the ADM had “legislation regulations or other legislative authorisations”.
Additionally, 12 said they had authorisations regarding “organisational policy and procedures”, two said they had “ministerial direction or guidelines” authorisations, while only one said that its ADM had “no explicit authorisation/directive”.
Be the first to hear the latest developments in the cyber industry.
