A powerful, high-performance CLI tool for removing duplicate lines from text files with advanced comparison options and parallel processing capabilities.
☆10Apr 16, 2025Updated 11 months ago
Alternatives and similar repositories for DupeRemover
Users that are interested in DupeRemover are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Continual Memorization of Factoids in Large Language Models☆12Nov 20, 2024Updated last year
- ☆25Dec 12, 2025Updated 3 months ago
- ☆17Aug 2, 2023Updated 2 years ago
- [ACL 2025 Main] Official Repository for "Evaluating Language Models as Synthetic Data Generators"☆41Dec 13, 2024Updated last year
- ☆54Oct 24, 2024Updated last year
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click and start building anything your business needs.
- ☆60Sep 24, 2024Updated last year
- Code for paper "Unraveling Cross-Modality Knowledge Conflicts in Large Vision-Language Models."☆53Oct 19, 2024Updated last year
- Repo for the paper "Large Language Models Struggle to Learn Long-Tail Knowledge"☆78Apr 12, 2023Updated 2 years ago
- Entity-Based Knowledge Conflicts in Question Answering. Code repo for EMNLP2021 paper: https://aclanthology.org/2021.emnlp-main.565/☆77Aug 29, 2022Updated 3 years ago
- The official code of TACL 2021, "Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategies".☆85Oct 31, 2022Updated 3 years ago
- ☆187Jul 2, 2025Updated 8 months ago
- LLM Unlearning☆182Oct 20, 2023Updated 2 years ago
- Code for the paper "Rethinking Benchmark and Contamination for Language Models with Rephrased Samples"☆320Dec 20, 2023Updated 2 years ago
- Scaling Data-Constrained Language Models☆342Jun 28, 2025Updated 8 months ago
- Virtual machines for every use case on DigitalOcean • AdGet dependable uptime with 99.99% SLA, simple security tools, and predictable monthly pricing with DigitalOcean's virtual machines, called Droplets.
- A project to improve skills of large language models☆882Updated this week
- LongBench v2 and LongBench (ACL 25'&24')☆1,122Jan 15, 2025Updated last year
- A bibliography and survey of the papers surrounding o1☆1,213Nov 16, 2024Updated last year
- This repo contains the source code for RULER: What’s the Real Context Size of Your Long-Context Language Models?☆1,487Nov 13, 2025Updated 4 months ago
- A reading list on LLM based Synthetic Data Generation 🔥☆1,531Jun 5, 2025Updated 9 months ago
- Everything competitive programming related - introductory guide, topics/concepts, practice problems, snippets & templates, tips & tricks …☆996Aug 29, 2023Updated 2 years ago
- ☆4,110Jun 4, 2024Updated last year
- Meta Lingua: a lean, efficient, and easy-to-hack codebase to research LLMs.☆4,752Jul 18, 2025Updated 8 months ago
- Efficient cache for gigabytes of data written in Go.☆8,106Feb 6, 2026Updated last month
- End-to-end encrypted cloud storage - Proton Drive • AdSpecial offer: 40% Off Yearly / 80% Off First Month. Protect your most important files, photos, and documents from prying eyes.
- A curated list of data engineering tools for software developers☆8,402Feb 21, 2026Updated last month
- Python scraper based on AI☆23,123Updated this week
- Structured Outputs☆13,588Updated this week
- dbt enables data analysts and engineers to transform their data using the same practices that software engineers use to build application…☆12,429Updated this week
- Algorithm and data structure articles for https://cp-algorithms.com (based on http://e-maxx.ru)☆10,314Mar 19, 2026Updated last week
- Official repository of Trino, the distributed SQL query engine for big data, formerly known as PrestoSQL (https://trino.io)☆12,662Updated this week
- A pattern-based approach to learn technical interview questions☆11,487Mar 17, 2026Updated last week
- Roadmap to becoming a data engineer in 2021☆12,745Jan 25, 2022Updated 4 years ago
- GoDS (Go Data Structures) - Sets, Lists, Stacks, Maps, Trees, Queues, and much more☆17,410Mar 12, 2025Updated last year
- NordVPN Special Discount Offer • AdSave on top-rated NordVPN 1 or 2-year plans with secure browsing, privacy protection, and support for for all major platforms.
- 🦔 Fast, lightweight & schema-less search backend. An alternative to Elasticsearch that runs on a few MBs of RAM.☆21,162Feb 17, 2026Updated last month
- The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lak…☆20,918Updated this week
- A Go microservices framework☆22,722Mar 16, 2026Updated last week
- SeaweedFS is a distributed storage system for object storage (S3), file systems, and Iceberg tables, designed to handle billions of files…☆31,065Updated this week
- Curated List of React Components & Libraries.☆47,125Jan 26, 2026Updated last month
- The fastest HTTP/2 Go Web Framework. New, modern and easy to learn. Fast development with Code you control. Unbeatable cost-performance r…☆25,615Jan 15, 2026Updated 2 months ago
- DSPy: The framework for programming—not prompting—language models☆33,038Updated this week