Enabling secure and effective near-duplicate detection over encrypted in-network storage

Near-duplicate detection (NDD) plays an essential role for effective resource utilization and possible traffic alleviation in many emerging network architectures, leveraging in-network storage for various content-centric services. As innetwork storage grows, data security has become one major concern. Though encryption is viable for in-network data protection, current techniques are still lacking for effectively locating encrypted near-duplicate data, making the benefits of NDD practically invalidated. Besides, adopting encrypted innetwork storage further complicates the user authorization when locating near-duplicate data from multiple content providers under different keys. In this paper, we propose a secure and effective NDD system over encrypted in-network storage supporting multiple content providers. Our design bridges locality-sensitive hashing (LSH) with a newly developed cryptographic primitive, multi-key searchable encryption, which allows the user to send only one encrypted query to access near-duplicate data encrypted under different keys. It relieves the users from multiple rounds of interactions or sending multiple different queries respectively. As simply applying LSH does not ensure the detection quality, we then leverage Yao's garbled circuits to build a secure protocol to obtain highly accurate results, without user-side post-processing. We formally analyze the security strength. Experiments demonstrate our system achieves practical performance with comparable accuracy to plaintext.