A powerful, high-performance CLI tool for removing duplicate lines from text files with advanced comparison options and parallel processing capabilities.
☆10May 25, 2026Updated 2 months ago
Alternatives and similar repositories for DupeRemover
Users that are interested in DupeRemover are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- Continual Memorization of Factoids in Large Language Models☆12Nov 20, 2024Updated last year
- [ACL 2025 Main] Official Repository for "Evaluating Language Models as Synthetic Data Generators"☆41Dec 13, 2024Updated last year
- Code associated to papers on superposition (in ML interpretability)☆40Sep 13, 2022Updated 3 years ago
- ☆54Oct 24, 2024Updated last year
- Code for paper "Unraveling Cross-Modality Knowledge Conflicts in Large Vision-Language Models."☆54Oct 19, 2024Updated last year
- Open source password manager - Proton Pass • AdSecurely store, share, and autofill your credentials with Proton Pass, the end-to-end encrypted password manager trusted by millions.
- Repo for the paper "Large Language Models Struggle to Learn Long-Tail Knowledge"☆77Apr 12, 2023Updated 3 years ago
- A dataset of over 10000 question and answer pairs written for storybooks.☆50May 31, 2023Updated 3 years ago
- ☆192Jul 2, 2025Updated last year
- Code for the paper "Rethinking Benchmark and Contamination for Language Models with Rephrased Samples"☆325Dec 20, 2023Updated 2 years ago
- Scaling Data-Constrained Language Models☆344Jun 28, 2025Updated last year
- [ICLR 2024] Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruning☆642Mar 4, 2024Updated 2 years ago
- LongBench v2 and LongBench (ACL 25'&24')☆1,214Jan 15, 2025Updated last year
- A project to improve skills of large language models☆1,012Jul 17, 2026Updated last week
- A bibliography and survey of the papers surrounding o1☆1,214Jul 7, 2026Updated 2 weeks ago
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- This repo contains the source code for RULER: What’s the Real Context Size of Your Long-Context Language Models?☆1,588Updated this week
- A reading list on LLM based Synthetic Data Generation 🔥☆1,544Jun 5, 2025Updated last year
- Freeing data processing from scripting madness by providing a set of platform-agnostic customizable pipeline processing blocks.☆3,220Updated this week
- ☆4,120Apr 15, 2026Updated 3 months ago
- Meta Lingua: a lean, efficient, and easy-to-hack codebase to research LLMs.☆4,757Jul 18, 2025Updated last year
- Python scraper based on AI☆28,610Updated this week
- A realtime distributed messaging platform☆25,759Updated this week
- ☆103,990Aug 28, 2025Updated 10 months ago
- Scalable datastore for metrics, events, and real-time analytics☆31,652Updated this week
- End-to-end encrypted email - Proton Mail • AdSpecial offer: 40% Off Yearly / 80% Off First Month. All Proton services are open source and independently audited for security.
- Code for the paper "Language Models are Unsupervised Multitask Learners"☆25,020Aug 14, 2024Updated last year
- Extremely fast Query Engine for DataFrames, written in Rust☆39,089Updated this week
- OpenAssistant is a chat-based assistant that understands tasks, can interact with third-party systems, and retrieve information dynamical…☆37,381Aug 17, 2024Updated last year
- DSPy: The framework for programming—not prompting—language models☆36,346Updated this week
- DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.☆42,797Updated this week
- No fortress, purely open ground. OpenManus is Coming.☆57,587Feb 11, 2026Updated 5 months ago
- Distributed reliable key-value store for the most critical data of a distributed system☆52,024Updated this week
- For developers, who are building real-time data-driven applications, Redis is the preferred, fastest, and most feature-rich cache, data s…☆75,659Updated this week
- The Go programming language☆135,338Updated this week
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- How to prepare audio and text to create a custom voice☆11Jul 14, 2020Updated 6 years ago
- ☆240Apr 3, 2026Updated 3 months ago
- This repository contains the resources and exercises for you when joining our team.☆27Jul 28, 2025Updated 11 months ago
- This is a comfyui project for expanding the prompt word or simple question and answer by ollama☆83Sep 17, 2024Updated last year
- Workspace for storing data through Resource.☆10Apr 15, 2024Updated 2 years ago
- Include only certain items in an array☆19Jun 13, 2025Updated last year
- Try OmniParser inComfyUI which a simple screen parsing tool towards pure vision based GUI agent.☆39Mar 12, 2025Updated last year