“The model pursued an authorized objective through an unauthorized path, crossed from a simulated environment into real companies and gained access without consent,” he said. “This is another area where regulations haven’t kept up with the pace of technology change. There are two sides to this: regulations with respect to the agentic escape and intrusion, and then the regulatory issues for the victim companies in terms of their requirements for disclosure. Google is only responsible for one half of that equation.”
Erik Avakian, technical counselor at Info-Tech Research Group, also noted that it’s critical that companies have rules about when to disclose unexpected and problematic model behaviors.
“I don’t think every unexpected thing an AI model does needs to become a public incident. But there should be a clear line once an autonomous system crosses an authorization or trust boundary,” he said. “If an AI system leaves a controlled environment, accesses a real third-party production system, uses credentials, retrieves data, escalates privileges, or takes some other action that was never authorized, that should, at minimum, trigger disclosure to the affected organization along with a formal incident investigation.”
Read the full article here
