AWS Storage Blog
Author: Tamelly Lim
Tamelly is a Specialist Solutions Architect based in Singapore, helping customers architect and implement AWS-based solutions. She began her cloud journey in 2022 after graduating university, and enjoys helping customers while learning and building in the cloud. Outside of work, she enjoys sports and exploring new cultures through travel.
Identifying potential duplicate objects in Amazon S3
Update 6/6/2025: Added “Important considerations” section that calls out reliance on MD5 and updated title from “Managing duplicate…” to “Identifying potential…” for accuracy. When managing a large volume of data in a storage system, it is common for data duplication to happen. Data duplication in data management refers to the presence of multiple copies of […]