viking-sudo-rm / rusty-dawgLinks
Rust library for indexing and quickly searching large pretraining corpora
☆26Updated last week
Alternatives and similar repositories for rusty-dawg
Users that are interested in rusty-dawg are comparing it to the libraries listed below
Sorting:
- ☆39Updated 4 years ago
- Code and data for "A Systematic Assessment of Syntactic Generalization in Neural Language Models"☆28Updated 3 years ago
- Sampling-Based Minimum Bayes-Risk Decoding for Neural Machine Translation☆16Updated 2 years ago
- M2D2: A Massively Multi-domain Language Modeling Dataset (EMNLP 2022) by Machel Reid, Victor Zhong, Suchin Gururangan, Luke Zettlemoyer☆55Updated 2 years ago
- source code of NAACL2021 "PCFGs Can Do Better: Inducing Probabilistic Context-Free Grammars with Many Symbols“ and ACL2021 main conferenc…☆50Updated 2 months ago
- Rationales for Sequential Predictions☆40Updated 3 years ago
- Repo for ICML23 "Why do Nearest Neighbor Language Models Work?"☆57Updated 2 years ago
- Repository with code for MaChAmp: https://aclanthology.org/2021.eacl-demos.22/☆87Updated 2 weeks ago
- Query-focused summarization data☆41Updated 2 years ago
- Statistics on multilingual datasets☆17Updated 2 years ago
- Code for ACL 2022 paper "Expanding Pretrained Models to Thousands More Languages via Lexicon-based Adaptation"☆30Updated 3 years ago
- ☆29Updated last year
- Collection of academic works in natural language processing, computational linguistics, and computational cognitive science that study th…☆18Updated last year
- XCOPA: A Multilingual Dataset for Causal Commonsense Reasoning☆103Updated 4 years ago
- The evaluation pipeline for the 2024 BabyLM Challenge.☆31Updated 6 months ago
- Code for "Does syntax need to grow on trees? Sources of inductive bias in sequence to sequence networks"☆23Updated 5 years ago
- Code for the paper "Modelling Latent Translations for Cross-Lingual Transfer"☆17Updated 3 years ago
- ☆35Updated 11 months ago
- Code for Massive-scale Decoding for Text Generation using Lattices☆44Updated 2 years ago
- ☆21Updated 2 years ago
- ☆58Updated 3 years ago
- Follow the Wisdom of the Crowd: Effective Text Generation via Minimum Bayes Risk Decoding☆18Updated 2 years ago
- ☆16Updated last year
- Cross-lingual TRansfer Evaluation of Multilingual Encoders (XTREME)☆22Updated 5 years ago
- ☆13Updated 2 years ago
- Modular implementation of an AM dependency parser in AllenNLP.☆32Updated 11 months ago
- Chu-Lui-Edmonds decoding extracted from TurboParser☆13Updated 8 years ago
- Learning to Model Editing Processes☆26Updated 3 years ago
- The LM Contamination Index is a manually created database of contamination evidences for LMs.☆78Updated last year
- BLOOM+1: Adapting BLOOM model to support a new unseen language☆72Updated last year