☆122Jan 24, 2026Updated 9 months ago
Alternatives and similar repositories for infini-gram
Users that are interested in infini-gram are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- ☆62Sep 26, 2025Updated last year
- An unofficial implementation of the Infini-gram model proposed by Liu et al. (2024)☆33Jun 19, 2024Updated 2 years ago
- A Multilingual Keyboard Layout-Based Typo Generator☆19Nov 23, 2025Updated 11 months ago
- ☆12Dec 24, 2025Updated 10 months ago
- Conditional Random Fields implemented in pure Rust☆13Mar 17, 2025Updated last year
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- ☆16Mar 1, 2026Updated 8 months ago
- This repository presents the original implementation of Pretraining Data Detection for Large Language Models: A Divergence-based Calibrat…☆23May 21, 2025Updated last year
- Rust library for indexing and quickly searching large pretraining corpora☆31Oct 30, 2025Updated last year
- ☆20Oct 2, 2026Updated last month
- ☆19Nov 5, 2025Updated last year
- ☆23Nov 6, 2022Updated 4 years ago
- ☆16Aug 7, 2026Updated 3 months ago
- Code base for the EMNLP 2021 Findings paper: Cartography Active Learning☆14Jun 3, 2025Updated last year
- Code for the paper "BPE stays on SCRIPT", "Which Pieces Does Unigram Tokenization Really Need?" and MinGram☆22Updated this week
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- Insights from the ICLR Peer Review and Rebuttal Process☆18Nov 24, 2025Updated 11 months ago
- A unified framework for attributing model behavior to model components, training data, and training dynamics.☆12Oct 25, 2026Updated last week
- ☆24Jun 16, 2026Updated 4 months ago
- ☆93Aug 18, 2024Updated 2 years ago
- [NAACL 2024] A Framework aims to wisely initialize unseen subword embeddings in PLMs for efficient large-scale continued pretraining☆18Nov 26, 2023Updated 2 years ago
- ☆109Jun 2, 2025Updated last year
- Metric calculation library☆11Jul 3, 2025Updated last year
- [ACL 2025] TokAlign: Efficient Vocabulary Adaptation via Token Alignment☆12Jun 4, 2025Updated last year
- ☆15Sep 30, 2025Updated last year
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- Repo for ACL2023 paper "Won't Get Fooled Again: Answering Questions with False Premises"☆23Jun 11, 2023Updated 3 years ago
- Code for the paper "Closing the Curious Case of Neural Text Degeneration"☆12Apr 9, 2025Updated last year
- Repo accompanying our paper "Do Llamas Work in English? On the Latent Language of Multilingual Transformers".☆88Mar 11, 2024Updated 2 years ago
- Code for Zero-Shot Tokenizer Transfer☆147Jan 14, 2025Updated last year
- [SIGMOD 2024] CAFE: Towards Compact, Adaptive, and Fast Embedding for Large-scale Recommendation Models☆13Mar 19, 2025Updated last year
- ☆12Oct 10, 2026Updated 3 weeks ago
- A collection of Pythonic subword tokenisers and text preprocessing tools.☆15Oct 29, 2026Updated last week
- A language server implementation for pysen☆10Nov 14, 2021Updated 4 years ago
- Code for Looped Transformers for Length Generalization (https://arxiv.org/abs/2409.15647), ICLR 2025☆15Apr 30, 2025Updated last year
- AI Agents on DigitalOcean Gradient AI Platform • AdBuild production-ready AI agents using customizable tools or access multiple LLMs through a single endpoint. Create custom knowledge bases or connect external data.
- [ACL 2023] Glot500: Scaling Multilingual Corpora and Language Models to 500 Languages☆108Apr 14, 2026Updated 6 months ago
- ☆18Jan 27, 2025Updated last year
- Repository for the paper: Aligning LLMs to Ask Good Questions A Case Study in Clinical Reasoning☆18Feb 21, 2025Updated last year
- Code for Retrofitting Large Language Models with Dynamic Tokenization.☆14Jul 22, 2025Updated last year
- CatIss is an intelligent tool for automatic categorization of issue reports based on the RoBERTa model.☆12Mar 8, 2022Updated 4 years ago
- What's In My Big Data (WIMBD) - a toolkit for analyzing large text datasets☆229Nov 16, 2024Updated last year
- This repo is a very minimal dependancy-free implementation (120 LOC) of SmolLMv2☆11Jan 29, 2025Updated last year