Google DeepMind has introduced a double‑blind evaluation workflow that runs Gemini 2.5 Flash Lite inside a hardware‑encrypted enclave, keeping the model weights hidden from the benchmark provider and the test prompts hidden from Google. This arrangement matters to AI, cloud, DevOps, and security engineers because it offers a concrete way to mitigate benchmark leakage while exposing new trust boundaries in the evaluation stack.
How the Confidential Enclave Works
The pilot combines Google Cloud Confidential Space with an NVIDIA H100 Confidential GPU and Intel TDX host‑memory encryption. Google supplies the Gemini model weights and inference code; the evaluator supplies the benchmark prompts and scoring scripts. Both parties transmit their assets over encrypted channels into the enclave, where the model runs and the prompts are evaluated. Remote attestation confirms that the enclave is running the agreed‑upon software before any data is loaded.
Performance Impact and Scaling Limits
The researchers measured less than 5 % overhead for the entire evaluation, indicating that the encryption and attestation steps do not dominate compute cost. The initial test ran on a single H100 80 GB Confidential GPU, which is sufficient for the Flash Lite variant but not for larger Gemini models. The team is already exploring multi‑GPU clusters linked with encrypted fabric to handle models that exceed a single GPU’s memory.
Trust Shifts and Security Considerations
While the enclave removes direct visibility of weights and prompts, it does not eliminate trust requirements. Practitioners must still trust the hardware manufacturer, the cloud provider, and the integrity of the enclave OS build, which relies on private signing keys. Legal agreements and pre‑approved code—facilitated by tools like OpenMined’s PySyft—are used to block network access from the evaluation code, preventing accidental data exfiltration. The paper notes that code reviews and contractual safeguards are currently larger hurdles than the measured compute overhead.
Related CloudNinjas coverage: AI engineering.
What This Means For Practitioners
Engineers evaluating closed‑source models should consider confidential enclaves when benchmark leakage threatens result validity. The approach suggests a pattern: keep proprietary assets (weights or prompts) inside a mutually attested, encrypted runtime, and rely on code‑level approvals to enforce data‑flow restrictions. When planning deployments, factor in the need for hardware that supports confidential execution (e.g., H100 Confidential GPUs) and the operational overhead of managing attestation reports and legal contracts. Monitoring the evolution of multi‑GPU confidential clusters will be important for scaling this technique to larger models.
