There are a variety of security concerns about artificial intelligence (AI), especially when it comes to the behavior of agentic AI. But until recently, the concept of locking down the models to prevent tampering hasn’t gotten a lot of attention.
Now, a security technology called “confidential computing” has emerged that could help solve that problem: it protects AI models from hackers by restricting models to authorized users. (It also protects data wherever it is — in storage, when moving between systems, and when it is accessed.)
With many top cloud and hardware providers championing confidential computing for AI, Computerworld talked with Dion Harris, Nvidia’s senior director of high-performance computing and AI factory solutions, about what the technology does and how it works.
Dion Harris, Nvidia’s senior director of high-performance computing and AI factory solutions.
Nvidia
Why should organizations care about confidential computing now? “Seventy percent of data exists outside the cloud, in on-premise data centers and data lakes. To deliver AI on this data while maintaining security, you need confidential computing to unlock that use case for enterprise AI.”
Why is confidential computing suddenly important in the AI age? “Enterprises want AI on sensitive data — customer records, medical information, financial data — without exposing it in cloud environments where they lose control.
“Traditional encryption protects data at rest and in transit, but not during computation. When you run an AI model, you must decrypt data. It sits in plain text in memory, accessible to administrators and cloud operators. Confidential computing creates a hardware-rooted trusted zone where data is decrypted only when computation needs it, then immediately re-encrypted.
“This lets enterprises get AI value without compromising security. Financial services, healthcare, government, and regulated industries are adopting it.”
How is confidential computing changing with agentic AI? “We’ve gone from generative to agentic AI being deployed and used to solve real business problems. To deliver agentic capabilities with required privacy, security and performance in enterprise, confidential computing provides the unlock.
“We can spawn agents, access data, leverage tools and generate real useful work. With agentic AI, confidential computing helps in two ways: protecting the data and helping design implementation and workloads. It’s deployment, implementation, and use combined.”
How does confidential computing work? “Encryption at rest protects stored data. Encryption in transit protects data moving over networks. Encryption in use is the problem — when you compute on data, you must decrypt it in memory.
“Confidential computing encrypts data in memory and between CPUs and GPUs. A dedicated element in the GPU decrypts information only when needed for computation, 100% inline, with minimal performance impact.”
Can you walk through a real-world example? “Apple’s standard operating policy for years has been to keep private information on device to avoid having access to private data. However, to leverage advanced AI models, they no longer fit on the device. Apple instituted their Private Compute Cloud, and now they’re extending that to Google Cloud.
“Let me give a hypothetical and hopefully describe how it works. If you have a user who wants to upload a medical transcript from their doctor, their system creates a secure, attested environment. It sends a request to the server: validate yourself. That attestation says: ‘I am a Nvidia GPU. This environment is secure. It hasn’t been tampered with.’ Now, it’s okay for you to send your information over that secure line. Remote attestation allows the edge device to ensure it’s sending to a trusted environment.
“The medical data comes over encrypted and remains encrypted until it lands in the GPU memory. Specific compute engines in the processor decrypt that information only to use it. The LLM dissects and summarizes it. Then it re-encrypts and sends it back over the wire.
“They get the capabilities of data center AI mode — larger, more advanced, delivering more services. But they also get the security and privacy from Apple’s [Private Cloud Compute] platform. It’s the best of both worlds: efficiencies and intelligence from data center AI with Apple’s PCC security.
What prevented confidential computing adoption? “The main challenge in the past was significant performance impact. When you adopted confidential computing, you had to trade off performance for privacy. A 30% to 40% throughput reduction undermined the economic viability of the entire solution. If you sacrifice that much performance, it reduces the ability to get full utilization out of the hardware you’re deploying to deliver tokens or deliver a service in an economical fashion.”
How was that solved? “With Blackwell and newer GPU architecture, you can deploy confidential computing without impact to performance. You get both privacy and performance, a win-win scenario.
“Performance now translates directly into economics. You get full utilization of the hardware, which matters for delivering tokens or services at scale. When we built Blackwell, we made confidential computing a first-class system feature to deliver not just the security, but also the performance the market requires.”
Where is adoption accelerating? “Companies are thinking about the cloud. Model builders can’t expose APIs, enterprises customize for their business. Hybrid on-prem and cloud models — where builders deliver services behind enterprise walls — require zero trust. We’re no longer in fully owned infrastructure.
“By 2030, [billions of dollars] is expected in confidential computing use cases. It’s emerging as essential infrastructure for AI adoption across the industry. For organizations using cloud infrastructure, deploying AI on sensitive data, or operating under regulatory requirements, confidential computing is becoming essential.”
What should organizations do? ”The performance problem is solved. The technology is ready, ecosystems are built, partners enable it. If you want to deploy agentic AI on sensitive data, protect customer privacy, meet regulatory requirements, and maintain security across hybrid environments, you need confidential computing in your strategy.
“Most customers start with a developer license, validate performance and security, then migrate to a commercial license. Partners like Red Hat and Fortanix integrate these mechanisms within their platforms. Google Cloud offers it through their services.
“Start with a proof of concept. Validate it works. Plan deployment. Organizations gain competitive advantage securing AI on sensitive data.”
Read the full article here
