Video Deduplication Using Clustering and Hashing-Based Layered Coarse Resolution Approach for Cloud Storage

Abstract

Managing storage effectively is crucial in the modern era of growing video data on cloud systems. The exponential increase in video content demands innovative solutions to manage storage space without compromising data integrity and access speed. Video deduplication techniques help to address the issue related to storage efficiency. Existing deduplication approaches either focus on computationally demanding deep learning techniques that limit deployment in resource-constrained situations or employ classical hashing to target precise duplication. This paper integrates clustering and hashing techniques at two resolution layers for video deduplication to maximize cloud-based storage efficiency toward reducing redundant data and improving system performance. Two resolution layers are clustering and hashing. Clustering layer groups similar videos based on video meta-parameters such as frame count and frames per second. Each cluster maintains metaparameters based on its videos. These cluster meta-parameters are compared with meta-parameters of the query video that comes to cloud storage. The matched cluster is considered as flagged cluster. Hash value of each video in the cluster is also maintained. Hashing layer compares each video hash value of the flagged cluster with query video hash value for deduplicate check using hashing-based similarity check logic. This layered resolution approach not only enhances accuracy but also significantly reduces the computational load and time required for deduplication. This approach not only supports scalable and cost-effective video storage solutions but also ensures data integrity and seamless access to multimedia content with improved resource management and user experience.