OpenAI agent accessed “credentials” via Medicare data portal
OpenAI agent accessed "credentials" via Medicare data portal
OpenAI has admitted the model that gained non-public access to a Medicare statistics portal also ran commands and retrieved credentials, going beyond its earlier account that only aggregate health statistics and internal file names were accessed.
An OpenAI blog post reveals additional details about the agent’s access to the now-offline Medicare statistics reporting portal, as well as to three other Australian government web-based resources.
OpenAI said that it tasked the agent that hit the Medicare data portal with “research[ing] government spending per person on medicines for skin conditions in Victorian communities.”
“The model had difficulty obtaining that information, and it took actions that we had not authorised it to take,” OpenAI wrote.
“In the course of looking for this information at Services Australia’s Medicare statistics reporting service, it discovered a way to gain non-public access to the service.
“It then used this access to review technical system information and source code related to the service – all still with the objective of trying to find the information it was originally looking for.”
OpenAI said that after gaining “non-public access to the service”, the agent “ran commands, retrieved internal files, credentials and aggregate statistics, and wrote files.”
A Services Australia spokesperson declined to detail what OpenAI’s model did, saying its forensic investigation with the Australian Signals Directorate is still establishing “the actions undertaken by the agent”.
Neither OpenAI nor Services Australia has said whose credentials the model retrieved, what systems they could unlock, or whether they have since been revoked.
Depending on what they were, the credentials could have given access to other government systems, which is why revoking them would normally be a priority.
In addition to providing information on the Medicare data portal interaction, OpenAI offered explanations for how its agent interacted with other mostly state-based web properties.
The OpenAI agent tasked with finding health information relating to Victoria also attempted to collate state-based health statistics for that purpose, but ran into problems doing so.
In the process, OpenAI said its agents “discovered an exposed access key to query the Victorian Agency for Health Information’s reporting system and retrieve reporting configuration and aggregate survey statistics”.
VAHI is part of the Victorian Department of Health.
OpenAI did not say how the key was exposed, and suggested it was up to VAHI to say whether its agents should have had that access.
“The extent to which this information should have been accessible is unclear, and depends on VAHI’s access policies,” OpenAI said.
Agents also sought out statistics from the Australian Institute of Health and Welfare (AIHW).
They were successful at getting public-facing data, but “attempts to bypass access controls were unsuccessful,” OpenAI noted.
OpenAI said the AIHW activity “did not meet our disclosure thresholds” because it seemed consistent with public access, but it notified the institute on 24 September anyway.
Crime numbers
OpenAI also said a model also made application programming interface (API) and website metadata requests through the public Crime Mapping Tool run by the NSW Bureau of Crime Statistics and Research (BOCSAR).
The tool, it said, “supplies credentials for browser API requests”.
“The BOCSAR system returned application configuration, operational jobs and logs, and website metadata,” OpenAI said.
OpenAI said crime records of individuals were not accessed.
BOCSAR initially said OpenAI had identified “a potential vulnerability that could allow access to the dataset underpinning BOCSAR’s Crime Mapping Tool”, without describing it.
A day later, the bureau said its investigations had found “no evidence of a security vulnerability” in the tool, and that no system changes had been identified as necessary.
It is not clear whether BOCSAR regards the tool returning configuration data and logs as intended behaviour.
Pauses model training
Chief strategy officer of OpenAI Jason Kwon is due to face questions about the incidents when he appears before the Joint Select Committee on Artificial Intelligence in Sydney on October 6.
OpenAI said it has “paused training and evaluation involving tool use for our most capable models”, and will resume only when it is confident additional safeguards are in place.
The company disclosed the pause in an earlier misalignment report describing a model that gained live internet access during a training run, using the domain name system (DNS) to reach an external chatbot.
About Author
.jpg "Australia Post signs three-year deal with Telstra")
.jpg "Protecting the Human Operating System: How Technology Teams Can Perform Under Pressure in an Always-On World")
.png "From ambition to impact: The next era of government transformation.")
