A powerful, high-performance CLI tool for removing duplicate lines from text files with advanced comparison options and parallel processing capabilities.
☆10May 25, 2026Updated 2 months ago
Alternatives and similar repositories for DupeRemover
Users that are interested in DupeRemover are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Continual Memorization of Factoids in Large Language Models☆12Nov 20, 2024Updated last year
- ☆26Dec 12, 2025Updated 8 months ago
- ☆17Aug 2, 2023Updated 3 years ago
- ☆35Jun 29, 2022Updated 4 years ago
- [ACL 2025 Main] Official Repository for "Evaluating Language Models as Synthetic Data Generators"☆42Dec 13, 2024Updated last year
- Deploy open-source AI quickly and easily - Special Bonus Offer • AdRunpod Hub is built for open source. One-click deployment and autoscaling endpoints without provisioning your own infrastructure.
- Code associated to papers on superposition (in ML interpretability)☆39Sep 13, 2022Updated 3 years ago
- ☆27Jun 4, 2026Updated 2 months ago
- Repo for the paper "Large Language Models Struggle to Learn Long-Tail Knowledge"☆77Apr 12, 2023Updated 3 years ago
- Entity-Based Knowledge Conflicts in Question Answering. Code repo for EMNLP2021 paper: https://aclanthology.org/2021.emnlp-main.565/☆77Aug 29, 2022Updated 3 years ago
- ☆160Dec 2, 2024Updated last year
- ☆193Jul 2, 2025Updated last year
- LLM Unlearning☆188Oct 20, 2023Updated 2 years ago
- Code for the paper "Rethinking Benchmark and Contamination for Language Models with Rephrased Samples"☆325Dec 20, 2023Updated 2 years ago
- Curate better data for LLMs☆1,072Mar 19, 2024Updated 2 years ago
- Deploy open-source AI quickly and easily - Special Bonus Offer • AdRunpod Hub is built for open source. One-click deployment and autoscaling endpoints without provisioning your own infrastructure.
- LongBench v2 and LongBench (ACL 25'&24')☆1,221Jan 15, 2025Updated last year
- A bibliography and survey of the papers surrounding o1☆1,215Jul 7, 2026Updated last month
- This repo contains the source code for RULER: What’s the Real Context Size of Your Long-Context Language Models?☆1,603Jul 22, 2026Updated 3 weeks ago
- A reading list on LLM based Synthetic Data Generation 🔥☆1,549Jun 5, 2025Updated last year
- The Art of Debugging Open Book☆1,686Updated this week
- Freeing data processing from scripting madness by providing a set of platform-agnostic customizable pipeline processing blocks.☆3,271Updated this week
- ☆4,122Apr 15, 2026Updated 3 months ago
- Meta Lingua: a lean, efficient, and easy-to-hack codebase to research LLMs.☆4,764Jul 18, 2025Updated last year
- Cybersecurity AI (CAI), the framework for AI Security☆9,726Jul 14, 2026Updated last month
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- Efficient cache for gigabytes of data written in Go.☆8,146Updated this week
- Best DDoS Attack Script Python3, (Cyber / DDos) Attack With 56 Methods☆16,540Updated this week
- Algorithm and data structure articles for https://cp-algorithms.com (based on http://e-maxx.ru)☆11,048Aug 2, 2026Updated last week
- A framework for few-shot evaluation of language models.☆13,631Updated this week
- Structured Outputs☆15,613Updated this week
- A curated list of awesome Competitive Programming, Algorithm and Data Structure resources☆14,114Dec 8, 2024Updated last year
- GoDS (Go Data Structures) - Sets, Lists, Stacks, Maps, Trees, Queues, and much more☆17,452Mar 12, 2025Updated last year
- Open-source data movement for ELT pipelines and AI agents — from APIs, databases & files to warehouses, lakes, and AI applications. Both …☆21,891Updated this week
- lightweight, idiomatic and composable router for building Go HTTP services☆22,670Updated this week
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- A realtime distributed messaging platform☆25,772Updated this week
- A Go agent harness and service framework☆23,011Updated this week
- SeaweedFS is a distributed storage system for object storage (S3), file systems, and Iceberg tables, designed to handle billions of files…☆34,063Updated this week
- The fastest HTTP/2 Go Web Framework. New, modern and easy to learn. Fast development with Code you control. Unbeatable cost-performance r…☆25,563Jul 27, 2026Updated 2 weeks ago
- 🧠 Laws, Theories, Principles and Patterns for developers and technologists.☆27,253May 19, 2026Updated 2 months ago
- CockroachDB — the cloud native, distributed SQL database designed for high availability, effortless scale, and control over data placemen…☆32,389Aug 7, 2026Updated last week
- GitHub’s official command line tool☆45,872Updated this week