Temporary remove unused tokens during training to save ram and speed.
☆23Jun 15, 2025Updated last year
Alternatives and similar repositories for transformer-smaller-training-vocab
Users that are interested in transformer-smaller-training-vocab are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- LV-BERT: Exploiting Layer Variety for BERT (Findings of ACL 2021)☆19May 10, 2023Updated 3 years ago
- ☆21Dec 5, 2022Updated 3 years ago
- ☆28Apr 19, 2026Updated 6 months ago
- ACL 2023 research paper☆10Sep 4, 2023Updated 3 years ago
- On the Similarities of Embeddings in Contrastive Learning (ICML 2025)☆11Jul 15, 2025Updated last year
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- ☆12Dec 24, 2025Updated 10 months ago
- ☆11Jan 27, 2025Updated last year
- ☆12Oct 10, 2026Updated 3 weeks ago
- Distil low resource language knowledge from large to small language models☆12Jan 28, 2026Updated 9 months ago
- ☆11Apr 6, 2026Updated 6 months ago
- Get plain text from Wikipedia pages☆11Jul 2, 2025Updated last year
- 🚀🤗 A collection of templates for Hugging Face Spaces☆34Oct 9, 2023Updated 3 years ago
- Fast brown clustering using python☆10Aug 21, 2023Updated 3 years ago
- Code for the paper "Getting the most out of your tokenizer for pre-training and domain adaptation"☆22Feb 14, 2024Updated 2 years ago
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- This repository provides code for the paper "Memory-Efficient Fine-Tuning of Transformers via Token Selection" (EMNLP 2024).☆14Feb 11, 2025Updated last year
- Research code for the paper "How Good is Your Tokenizer? On the Monolingual Performance of Multilingual Language Models"☆27Oct 3, 2021Updated 5 years ago
- CNER: Concept and Named Entity Recognition☆13Jul 15, 2024Updated 2 years ago
- [ACL 2025] 🔍 Multilingual Evaluation of English-Centric LLMs via Cross-Lingual Alignment☆11Apr 6, 2025Updated last year
- Implementation of Cascaded Head-colliding Attention (ACL'2021)☆11Sep 16, 2021Updated 5 years ago
- [ICML 2026] Improving GPT via a simple normalization strategy☆15May 22, 2026Updated 5 months ago
- Minimal code to train ELMo models in recent versions of TensorFlow☆14Jun 16, 2026Updated 4 months ago
- ☆11Mar 17, 2026Updated 7 months ago
- Getting interpretable dimensions in word embedding spaces.☆15Jul 6, 2023Updated 3 years ago
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- A Multilingual Keyboard Layout-Based Typo Generator☆18Nov 23, 2025Updated 11 months ago
- ChunkNorris is a black belt in document chunking to feed your LLMs and RAG apps 🥋🔪☆30Oct 26, 2026Updated last week
- Resources related to EMNLP 2021 paper "FAME: Feature-Based Adversarial Meta-Embeddings for Robust Input Representations"☆13Dec 14, 2021Updated 4 years ago
- A framework for adversarial attacks against token classification models☆33Nov 6, 2021Updated 4 years ago
- Python wrapper around uroman tokenizer☆12Oct 12, 2023Updated 3 years ago
- PyTorch-IE: State-of-the-art Information Extraction in PyTorch☆76Sep 24, 2025Updated last year
- A set of classes for calculating cryptocurrency tax (US)☆10Feb 8, 2018Updated 8 years ago
- Code for the paper "BPE stays on SCRIPT", "Which Pieces Does Unigram Tokenization Really Need?" and MinGram☆22Oct 21, 2026Updated last week
- Maximum entropy named-entity recognition (NER)☆13Dec 8, 2022Updated 3 years ago
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- A simple library for training named entity recognition model from partially annotated data☆24Nov 12, 2023Updated 2 years ago
- SpeechGLUE is a speech version of the GLUE benchmark, driven by text-to-speech.☆13Jun 2, 2023Updated 3 years ago
- FlexiTokens☆23Dec 27, 2025Updated 10 months ago
- A curated list of tokenizer libraries for blazing-fast NLP processing.☆14Apr 10, 2025Updated last year
- [ACL'26 Findings] Recovered in Translation: Efficient Pipeline for Automated Translation of Benchmarks and Datasets☆20Jun 27, 2026Updated 4 months ago
- State-of-the-Art Neural Coreference Resolution on German Data☆11Dec 16, 2022Updated 3 years ago
- BPE modification that implements removing of the intermediate tokens during tokenizer training.☆27Nov 25, 2024Updated last year