A powerful, high-performance CLI tool for removing duplicate lines from text files with advanced comparison options and parallel processing capabilities.
☆11May 25, 2026Updated 4 months ago
Alternatives and similar repositories for DupeRemover
Users that are interested in DupeRemover are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- [ACL 2025 Main] Official Repository for "Evaluating Language Models as Synthetic Data Generators"☆42Dec 13, 2024Updated last year
- Code for "Tracing Knowledge in Language Models Back to the Training Data"☆40Dec 27, 2022Updated 3 years ago
- ☆54Oct 24, 2024Updated last year
- ☆195Jul 2, 2025Updated last year
- Scaling Data-Constrained Language Models☆348Jun 28, 2025Updated last year
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- [CSUR 2025] Continual Learning of Large Language Models: A Comprehensive Survey☆567Dec 23, 2025Updated 9 months ago
- Curate better data for LLMs☆1,070Mar 19, 2024Updated 2 years ago
- LongBench v2 and LongBench (ACL 25'&24')☆1,247Jan 15, 2025Updated last year
- This repo contains the source code for RULER: What’s the Real Context Size of Your Long-Context Language Models?☆1,624Jul 22, 2026Updated 2 months ago
- Freeing data processing from scripting madness by providing a set of platform-agnostic customizable pipeline processing blocks.☆3,390Oct 6, 2026Updated last week
- Package macaron is a high productive and modular web framework in Go.☆3,545Feb 16, 2026Updated 8 months ago
- Meta Lingua: a lean, efficient, and easy-to-hack codebase to research LLMs.☆4,771Jul 18, 2025Updated last year
- A fault tolerant, protocol-agnostic RPC system☆8,868Aug 13, 2026Updated 2 months ago
- Efficient cache for gigabytes of data written in Go.☆8,170Updated this week
- Managed Database hosting by DigitalOcean • AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- Highly available Prometheus setup with long term storage capabilities. A CNCF Incubating project.☆14,238Updated this week
- A curated list of awesome big data frameworks, ressources and other awesomeness.☆14,677Jul 31, 2026Updated 2 months ago
- A framework for few-shot evaluation of language models.☆14,248Sep 14, 2026Updated last month
- Structured Outputs☆15,949Sep 21, 2026Updated 3 weeks ago
- ZincSearch . A lightweight alternative to elasticsearch that requires minimal resources, written in Go.☆17,919Sep 16, 2026Updated last month
- Vitess is a database clustering system for horizontal scaling of MySQL.☆21,388Updated this week
- Roadmap to becoming a Go developer in 2020☆18,417Feb 13, 2023Updated 3 years ago
- SeaweedFS is a distributed storage system for object storage (S3), file systems, and Iceberg tables, designed to handle billions of files…☆35,541Updated this week
- Distributed Task Queue (development branch)☆28,994Updated this week
- Virtual machines for every use case on DigitalOcean • AdGet dependable uptime with 99.99% SLA, simple security tools, and predictable monthly pricing with DigitalOcean's virtual machines, called Droplets.
- The fastest HTTP/2 Go Web Framework. New, modern and easy to learn. Fast development with Code you control. Unbeatable cost-performance r…☆25,555Jul 27, 2026Updated 2 months ago
- CockroachDB — the cloud native, distributed SQL database designed for high availability, effortless scale, and control over data placemen…☆32,574Updated this week
- GitHub’s official command line tool☆46,673Updated this week
- Shannon is an AI pentester for web applications and APIs. It analyzes your source code, identifies attack vectors, and executes real expl…☆49,041Updated this week
- Extremely fast Query Engine for DataFrames, written in Rust☆40,105Updated this week
- 🧙♀️ Move Fast and Break Nothing. End-to-end typesafe APIs made easy.☆40,711Updated this week
- OpenAssistant is a chat-based assistant that understands tasks, can interact with third-party systems, and retrieve information dynamical…☆37,373Aug 17, 2024Updated 2 years ago
- ✅ Solutions to LeetCode by Go, 100% test coverage, runtime beats 100% | LeetCode 题解☆33,811Updated this week
- DSPy: The framework for programming—not prompting—language models☆38,666Updated this week
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.☆44,051Updated this week
- No fortress, purely open ground. OpenManus is Coming.☆58,699Sep 30, 2026Updated 2 weeks ago
- Deep Learning papers reading roadmap for anyone who are eager to learn this amazing tech!☆39,585Nov 27, 2022Updated 3 years ago
- ClickHouse® is a real-time analytics database management system☆50,458Updated this week
- Cheatsheets for experienced React developers getting started with TypeScript☆47,099Sep 9, 2026Updated last month
- 🕷️ An adaptive Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: http…☆88,248Updated this week
- Apache Spark - A unified analytics engine for large-scale data processing☆44,230Updated this week