mk-fg / image-deduplication-tool
Tool to detect (and get rid of) similar images using perceptual hashing (pHash lib)
☆81Updated 8 years ago
Related projects ⓘ
Alternatives and complementary repositories for image-deduplication-tool
- Video fingerprinting tool. Finding duplicate movies in a large dataset.☆43Updated 12 years ago
- Serving content from a WARC☆60Updated 11 years ago
- Identifies similar pictures on your local computer☆78Updated 5 years ago
- Implementation of perceptual image hash calculation in Python☆130Updated last year
- Tools for bulk indexing of WARC/ARC files on Hadoop, EMR or local file system.☆42Updated 6 years ago
- Attempt to use perceptual hash (pHash) to segment a video into "scenes" very quickly (Normally under a minute for hour long HD videos).☆46Updated last year
- WarcMiddleware lets users seamlessly download a mirror copy of a website when running a web crawl with the Python web crawler Scrapy.☆44Updated 6 years ago
- Deep Zoom Image Downloader☆18Updated 6 months ago
- Perceptual Hash project for Videos (MMAI Term Project)☆27Updated 10 years ago
- A Python Perceptual Image Hashing Module☆209Updated 2 years ago
- Tool for managing data-deduplication within extant compressed archive files, along with a relatively performant BK tree implementation fo…☆98Updated last year
- A comparison of ffmpeg, Shotdetect and PySceneDetect for shot transition detection☆119Updated 6 years ago
- A Memento Aggregator CLI and Server in Go☆57Updated 6 months ago
- Tutorial on detecting video shot changes using Python and OpenCV. Part 1 covers basic threshold detection, Part 2 covers optimized thres…☆48Updated 4 years ago
- isk-daemon is an open source standalone server and library capable of adding content-based (visual) image searching to any image related …☆137Updated 9 years ago
- Command-line tool to easily extract data from HTML or XML documents. Produces machine readable output.☆31Updated 5 years ago
- Filmstrip is an OpenCV/Python based set of scripts for extracting keyframes from video.☆87Updated 7 years ago
- Modular workflow assistant for book digitization☆18Updated 8 years ago
- Python library for reading and writing warc files☆237Updated 2 years ago
- unified cli for various saas image classification apis.☆40Updated 7 years ago
- Extract .jpg files from video, with accurate frame rate manipulation☆15Updated 11 years ago
- Check out https://github.com/webrecorder/webrecorder for newer version matching https://webrecorder.io☆39Updated 9 years ago
- recursively deduplicate a directory and write its contents to a new directory while remembering the old paths☆47Updated 4 years ago
- Webrecorders DevTools Protocol Automation Library☆17Updated 2 years ago
- Identifying and removing near-duplicate images using perceptual hashing.☆350Updated last year
- a Python library to communicate with an instance of Phil Harvey's excellent ExifTool command-line application.☆271Updated last year
- Sort-friendly URI Reordering Transform (SURT) python module☆40Updated 3 months ago
- WARC writing MITM HTTP/S proxy☆381Updated 2 weeks ago
- Automated shot detection software☆199Updated last year
- A list of tools related to W(eb)ARC(hive)☆54Updated 10 years ago