duplicate-detection

Remove duplicates from MASSIVE wordlist, without sorting it (for dictionary-based password cracking)

⚡ Check your npm modules for unused and duplicate dependencies fast

Near Duplicate Video Detection (Perceptual Video Hashing) - Get a 64-bit comparable hash-value for any video.

The Panako acoustic fingerprinting system.

Fast Near-Duplicate Image Search and Delete using pHash, t-SNE and KDTree.

Filter, Sort & Delete Duplicate Files Recursively

CLI tool that fast checks if your bundle contains multiple versions of the same package, only by looking in package.json.

CLI utility to find near duplicate images and remove all but the best copy.

Find similar audio files easily

A collection of publicly available bug reports

Vidupe is a program that can find duplicate and similar video files. V1.211 released on 2019-09-18, Windows exe here:

Detecting near-duplicate videos by aggregating features from intermediate CNN layers

Interactive code for image similarity using SIFT algorithm

Duplicates finder for various source code formats.

Advanced Duplicate File Finder for Python

Nextcloud Media Duplicate Collector application

Advanced similarity and duplicate source code proof of concept for our research efforts.

Remove Duplicate Messages

Advanced similarity and duplicate source code at scale.

Wanna get DRY? Static analysis tool for detecting repeat code.