Identifying and removing near-duplicate images using perceptual hashing.

File Integrity Manager -

ECMAScript proposal for Deduplicating method of Array

JS script that allows you to remove duplicates from your Last.fm scrobbles library.

Benchmarking javascript array operations

A C++ reimplementation of Near Duplicate Video Detection - Get a 64-bit comparable hash-value for any video (Video Hash).

A funny extension that integrates image-browsing , downloader , deduplicate , cluster , can quickly collect, classify and process your images. | 一个有趣的扩展，整合了 图库，下载，去重，聚类 ，可以快速搜集、分类、处理你的图片。

Project to take two similar zipfiles, and to dedupe files that have the same tiemstamp in the older file.

😺一款用于防止重复攻击的组件，基于SPI机制实现核心功能 >>> 防重放组件 (A component used to prevent duplicated attacks, such as repeat request, replay attack)

A command-line tool for deduplicating entries in a file or stream with constant memory usage

Yet another tool to find and remove duplicate files.

Deduplication API for Google MusicManager

Go implementation of Facebook's DataLoader with 200+ lines of code.

Package uniq provides primitives for getting the first unique elements of (aka deduplicate) your existing sorted sort.Interface.

Deduplicates maildir contents using hard links.

A command line tool for sorting photo's from the Synology DSPhoto auto-upload tool

动漫一拍N自动识别算法

Returns duplicates-removed array -- has options for case-sensitivity and strict-typing.

makes any request idempotent across nodejs frameworks like nestjs, express, fastify

A shell script that deduplicates files on a filesystem