fake-name / IntraArchiveDeduplicator
Tool for managing data-deduplication within extant compressed archive files, along with a relatively performant BK tree implementation for fuzzy image searching.
☆100Updated last year
Alternatives and similar repositories for IntraArchiveDeduplicator:
Users that are interested in IntraArchiveDeduplicator are comparing it to the libraries listed below
- Implementation of perceptual image hash calculation in Python☆131Updated last year
- Github-Repository of the pHash.org library for perceptual hashing.☆222Updated 6 years ago
- ☆88Updated last year
- WarcMiddleware lets users seamlessly download a mirror copy of a website when running a web crawl with the Python web crawler Scrapy.☆46Updated 6 years ago
- A reverse image search algorithm which performs 2D affine transformation-invariant partial image-matching in sublinear time☆291Updated 5 years ago
- Tools for bulk indexing of WARC/ARC files on Hadoop, EMR or local file system.☆44Updated 7 years ago
- Identifies similar pictures on your local computer☆77Updated 5 years ago
- Tool to detect (and get rid of) similar images using perceptual hashing (pHash lib)☆82Updated 8 years ago
- An HTTP-based warc-to-zip converter☆11Updated 11 years ago
- A multi format lossless image optimizer that uses external tools☆111Updated this week
- A library for sending software performance metrics from Python libraries and apps to statsd.☆30Updated 8 months ago
- Python video metadata parser☆61Updated 3 weeks ago
- Saves proxied HTTP traffic to a WARC file.☆27Updated 11 years ago
- This is a demo project for libpuzzle. The project shows how to use sql-like database to index millions of images.☆14Updated 11 years ago
- search, dedupe, and media ingestion for mediachain☆33Updated 8 years ago
- Remote client for distributed automated HTTP(s) content fetching.☆78Updated this week
- A Docker image that contains a set of tools for compressing and optimize images in many formats with high results, it also includes GNU P…☆20Updated last year
- Converts YouTube XML Annotations to ASS subtitles☆16Updated 6 years ago
- A Python implementation of the Double Metaphone algorithm☆61Updated 14 years ago
- Hamming distance between hex strings in SQLite☆25Updated 7 years ago
- Fuse tag file system☆126Updated 5 years ago
- Fast text chunking algorithms for Python☆12Updated 4 years ago
- Unix de-duplicating archiver☆123Updated 10 years ago
- Encrypted backups (without the backups)☆121Updated 10 years ago
- Video fingerprinting tool. Finding duplicate movies in a large dataset.☆43Updated 12 years ago
- Neddick: Open Source Information Discovery Platform☆36Updated last year
- Fast Python bindings to Sophia Database☆80Updated 4 months ago
- pHash - the open source perceptual hash library☆576Updated 2 years ago
- templatemaker is a Python library that can extract data from files with a similar format, like HTML pages.☆63Updated 4 years ago
- image perceptual hash based on ML☆27Updated 2 years ago