Imagededup

This package provides functionality to make use of hashing algorithms that are particularly good at finding exact duplicates as well as convolutional neural networks which are also adept at finding near duplicates. An evaluation framework is also provided to judge the quality of deduplication for a given dataset.

this is great for finding duplicates, especially if you have masses of files, it's for images files and they have downloads that work cross platform on python 3.6+ -- pretty sweet, if you wanna do it the open source way.