#

dedupe

Here are 69 public repositories matching this topic...

restic / restic

Fast, secure, efficient backup program

go backup dedupe secure-by-default discover restic deduplication

Updated Jun 1, 2022
Go

borgbackup / borg

Open

debug log all cache / index full paths

ThomasWaldmann commented Aug 21, 2021

Considering issues like #5954 are bad to solve / debug without this information, borg should debug log full paths of all caches / indexes it uses.

Read more

enhancement easy good first issue

Open

using compression for files cache?

6

Open

re: patterns not accepted for prune

5

Find more good first issues

dedupeio / dedupe

🆔 A python library for accurate and scalable fuzzy matching, record deduplication and entity-resolution.

python clustering dedupe record-linkage python-library entity-resolution datamade dedupe-library de-duplicating

Updated Jun 2, 2022
Python

scinos / yarn-deduplicate

Deduplication tool for yarn.lock files

yarn dedupe duplicates lock-file yarn-lock duplicated-packages

Updated Jun 1, 2022
TypeScript

jbruchon / jdupes

A powerful duplicate file finder and an enhanced fork of 'fdupes'.

Updated Mar 18, 2022
C

blakeembrey / free-style

Sponsor

Make CSS easier and more maintainable by using JavaScript

javascript css typescript js dedupe css-in-js hash minification css-string

Updated Nov 9, 2021
TypeScript

recordlinkage

J535D165 / recordlinkage

Sponsor

A powerful and modular toolkit for record linkage and duplicate detection in Python

python machine-learning privacy dedupe record-linkage python-library entity-resolution similarity string-distance deduplication data-matching utrecht-university

Updated Apr 19, 2022
Python

duplicut

nil0x42 / duplicut

Sponsor

Remove duplicates from MASSIVE wordlist, without sorting it (for dictionary-based password cracking)

c dictionary dedupe password wordlist remove-duplicates uniq hashes cracking wordlist-generator hashcat unique duplicate-detection password-cracking wordlists

Updated Dec 17, 2021
C

zingg

zinggAI / zingg

Open

Add AGPL license header to all code.

sonalgoyal opened May 24, 2022

good first issue

Open

test date type

Open

rename "dont use" to dont_use consistent with other match types

1

Find more good first issues

dedupeio / csvdedupe

🆔 Command line tool for deduplicating CSV files

cli dedupe record-linkage csv-files entity-resolution

Updated Mar 31, 2020
Python

dedupeio / dedupe-examples

🆔 Examples for using the dedupe library

python dedupe record-linkage entity-resolution

Updated Jan 19, 2022
Python

Zygo / bees

Best-Effort Extent-Same, a btrfs dedupe agent

beer filesystem dedupe extent hash btrfs dedup

Updated May 14, 2022
C++

knjcode / imgdupes

Finding and deleting near-duplicate images based on perceptual hash.

image dedupe perceptual-hashing perceptual-hashes deduplicate

Updated Mar 11, 2022
Python

kornelski / dupe-krill

A fast file deduplicator

macos dedupe rust-library file-deduplication hardlinks dupes

Updated Jan 25, 2022
Rust

mail-deduplicate

kdeldycke / mail-deduplicate

Sponsor

📧 CLI to deduplicate mails from mail boxes.

python cli mail email dedupe mailbox cleanup maildir mbox deduplication mh babyl mmdf

Updated May 20, 2022
Python

Lakshmipathi / dduper

Open

Add test scripts for RAID config.

Lakshmipathi commented Aug 24, 2020

Right dduper has minimal test script to check basic functionality See ci/gitlab/*.sh . Enhance it add RAID tests.

Read more

good first issue qa

Open

Add travisci.yml for basic sanity test

dssg / pgdedupe

A simple command line interface to the datamade/dedupe library.

python database dedupe record-linkage postgresql data-cleaning deduplication

Updated Nov 15, 2021
Jupyter Notebook

jchristn / WatsonDedupe

Sponsor

Self-contained C# library for data deduplication using Sqlite

compression storage nuget dedupe sqlite-database data-deduplication chunk compress deduplication chunk-data duplicate-data chunk-key

Updated Jan 7, 2022
C#

jRimbault / yadf

Yet Another Dupes Finder

dedupe deduplication fdupes duplicate-detection file-deduplication dupes-finder

Updated Apr 7, 2022
Rust

mighty-justice / django-super-deduper

Utilities for de-duping Django model instances

python django dedupe

Updated Jul 30, 2021
Python

zirkonit / duplex

Duplicate code finder for Elixir

lint elixir escript static-code-analysis dedupe static-analysis duplex

Updated Apr 12, 2021
Elixir

kevinpollet / pocket-deduper

Remove duplicates from your Pocket list.

go cli golang tool dedupe duplicates pocket

Updated Jan 1, 2022
Go

dedupeio / dedupe-variable-address

Address Variable Type for dedupe

dedupe dedupe-variable

Updated Mar 31, 2020
Python

dedupeio / dedupe-variable-name

name variable type for dedupe

dedupe dedupe-variable

Updated Jan 11, 2022
Python

dedupeio / dedupe-variable-employer

dedupe dedupe-variable

Updated Mar 31, 2020
Python

dedupeio / dedupe-variable-fuzzycategory

Dedupe Variable for Fuzzy Categories

dedupe dedupe-variable

Updated Mar 31, 2020
Python

lilydjwg / android-dedupefs

A filesystem for reading Android dedupe backup

android fuse backup dedupe

Updated Jul 17, 2014
C

amoghe / dedup

Deduplicator

compression dedupe compressor deduplication

Updated Aug 9, 2017
Go

dedupeio / parseratorvariable

Base class for dedupe variables for parsed fields

dedupe dedupe-variable

Updated Mar 31, 2020
Python

apollo008 / indexed-priority-queue

indexed-priority-queue 是C++编程语言实现的开源的可以快速改变某个指定元素值的优先级队列。它通过为优先级队列中每个元素维护一个索引Index值来实现快速修改特定元素值且又保证堆性质的目的。

dedupe priority-queue indexed mutable topn

Updated Dec 21, 2021
C++

Improve this page

Add a description, image, and links to the dedupe topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with the dedupe topic, visit your repo's landing page and select "manage topics."