Google Built a Compiler That Runs AI on Data It Can Never See

TL;DR

On August 14, 2026, Google announced a maturity milestone for HEIR (Homomorphic Encryption Intermediate Representation), the open-source, Apache 2.0, MLIR-based compiler it has been building since 2023 to let ordinary AI models run directly on encrypted data. The pitch: a server can compute a recommendation, a fraud score, or a wake-word match without ever holding a key that could decrypt what it's processing. Google backed the claim with four working demos built alongside outside partners, not just a benchmark slide. The honest catch, stated in Google's own post, is that today's published numbers are single-threaded CPU only, and the custom hardware that's supposed to make this fast is still "in the near future."

Three Quiet Years, Then Four Demos at Once

HEIR isn't new. Google researcher Jeremy Kun first wrote about starting the project back in August 2023, as an attempt to give homomorphic encryption (HE) research a shared compiler instead of every lab hand-rolling its own. What changed this week is proof of use: instead of another paper, Google shipped four production-shaped applications built jointly with named outside companies, plus a paper arguing HEIR is becoming the de facto standard for the field. It's also collected four peer-reviewed publications and academic collaborators including Georgia Tech, Carnegie Mellon, UC Santa Barbara, Purdue, and Tsinghua University.

There's a wry irony sitting right at the center of this launch: the company whose core business runs on knowing exactly what you searched, watched, and clicked just spent three years building infrastructure whose entire purpose is to guarantee a server mathematically cannot know what it's processing. Whether that irony is the point or just a side effect is a question Google's blog post, understandably, does not address.

What Homomorphic Encryption Actually Buys You

Normal encryption protects data at rest and in transit, but the moment a server needs to actually compute on it, that data has to get decrypted first, if only in memory. Homomorphic encryption (HE) removes that step: it lets you perform math, addition, multiplication, the operations a neural network is built from, directly on ciphertext, and get back an encrypted result that decrypts to the same answer you'd have gotten on the plaintext.

Think of it like a locked box with built-in gloves sewn into the sides: whoever is holding the box can reach in and rearrange what's inside, weigh it, combine it with other things passed through similar boxes, all without ever unlocking it or seeing what's in it. Only the person holding the actual key, you, can open the box afterward and see the result. The server did real work; it just never got to look.

The Four Things It Actually Ran

Google's post lists four applications HEIR compiled with named partners, each picked to look like a real product feature rather than a toy benchmark:

• Content recommendations: a deep learning recommendation model built with Belfort Labs, LG, and New York University.
• Credit card fraud detection: compiled together with Niobium Microsystems and hardshell.ai.
• Network intrusion detection: the academic Kitsune anomaly-detection system, run on encrypted network traffic with Niobium.
• Hotword detection: a wake-word model with Belfort Labs, aimed at letting an audio-triggered AI agent listen for its name without ever handling raw audio.

That last one is the case worth sitting with if you build anything with an always-listening AI agent: the promise is a device that can recognize "hey assistant" without the recording ever existing anywhere as plaintext, including on the vendor's own servers.

The Catch: It's Not Fast Yet

HE has a well-earned reputation for being punishingly slow, historically many orders of magnitude slower than the same computation on plaintext, because every basic operation on ciphertext costs far more than its plaintext equivalent. Google's post is upfront that the latency numbers it published for these four demos are for single-threaded CPU execution only. The company says it's working with four hardware accelerator partners, Belfort, Niobium, Cornami, and Optalysys, and plans to demonstrate the speed benefits of dedicated silicon "in the near future," which is a careful way of saying that part hasn't shipped yet.

Read plainly, this week's announcement is a correctness milestone, not a speed one: proof that a compiler can take an ordinary model and produce a working encrypted version of it, across four realistic applications, with named partners willing to put their names on it. The performance story is still being written by four separate hardware startups.

Picking a Backend

HEIR doesn't implement cryptography itself, it compiles down to existing HE libraries, and which one you target depends on what your model actually does. OpenFHE and Lattigo both cover the lattice-based schemes, BGV, BFV, and CKKS, that are good at batched arithmetic like the matrix multiplies inside a neural network. tfhe-rs and Jaxite, Google's own JAX-based library, instead target CGGI (also known as TFHE), which is slower per-operation but handles branching logic that the lattice schemes choke on.

Why This Matters If You Build With Data You'd Rather Not Hold

Most self-hosted and indie-hacker AI projects solve the privacy problem by not sending data anywhere, running local models on your own hardware. HE is the answer for the opposite case: when you need a server, maybe because the model is too big to run locally, but you don't want that server, or whoever eventually subpoenas it, to ever see the plaintext. Fraud detection, health data, and biometric matching are the categories where regulators already care more than users do, and where "we mathematically cannot read your data" is a stronger claim than any privacy policy.

None of that changes what HN's discussion of Google's post kept circling back to: HE remains slow enough that most teams will keep reaching for local inference or plain old TLS-plus-trust for anything latency-sensitive. HEIR's actual contribution is lowering the expertise bar, from "hire a cryptographer" to "annotate which variables are secret and compile", not eliminating the tax you pay for using it.

Key Takeaways

• Google published a maturity milestone for HEIR on August 14, 2026, an open-source, Apache 2.0 compiler it has developed since 2023 for running AI models on homomorphically encrypted data
• Four named, production-shaped demos shipped with it: a recommendation model, a credit card fraud detector, network intrusion detection via Kitsune, and hotword detection, each built with an outside partner
• The published latency numbers are single-threaded CPU only; Google says hardware acceleration from four partners, Belfort, Niobium, Cornami, and Optalysys, is still to be demonstrated
• HEIR compiles to existing libraries, not its own cryptography: OpenFHE and Lattigo for lattice-based schemes (BGV/BFV/CKKS), tfhe-rs and Google's own Jaxite for CGGI
• The pitch is lowering the expertise bar, not eliminating the performance cost that has made homomorphic encryption impractical for most real-time workloads until now

Sources: Google: How Google is making private AI practical with homomorphic encryption, google/heir on GitHub, heir.dev, HEIR: A Universal Compiler for Homomorphic Encryption (arXiv), Jeremy Kun: Google's Recent FHE Work, and Starting HEIR, Hacker News discussion