Few-shot Semantic Recovery Attacks on Image Embeddings
Abstract
AbstractImage embeddings are an essential commodity in modern AI: stored at scale in vector databases, transferred between edge devices and cloud APIs, and traded as paid services. Image embeddings have been implicitly treated as anonymous—an assumption that has not been rigorously tested even as embedding services scale to billions of stored vectors. We show this assumption breaks. We propose Cross-modal Recovery Attack with Few-shot Transfer (CRAFT), a framework that recovers semantic meaning, including free-form captions and sensitive attributes. The attack focuses on the medical domain and facial features, targeting black-box image embeddings without reconstructing a single image in pixels and without any knowledge of the underlying model architecture, training data, or weights. CRAFT combines a public dual-encoder, a linear alignment learned from intercepted image-embedding pairs, and a fine-tuned embedding-to-text decoder; we further strengthen recovery with two augmentative techniques: cross-modal reranking against the attacker’s text encoder and chain-of-thought prompting on a diffusion-rendered low-fidelity reconstruction of the leaked embedding. Attacks succeed with as few as a single intercepted pair and transfer across five open-source and proprietary embedders. We disclose a privacy surface that current defenses cannot close. Every setting that preserves downstream utility also leaves the attack effective. Black-box image embeddings, as widely deployed in AI systems today, should not be treated as privacy-preserving artifacts.