owos / flexitokensLinks
FlexiTokens
☆18Updated last month
Alternatives and similar repositories for flexitokens
Users that are interested in flexitokens are comparing it to the libraries listed below
Sorting:
- State-of-the-art paired encoder and decoder models (17M-1B params)☆53Updated 3 months ago
- Starbucks: Improved Training for 2D Matryoshka Embeddings☆22Updated 4 months ago
- Trully flash implementation of DeBERTa disentangled attention mechanism.☆66Updated last month
- Code for SaGe subword tokenizer (EACL 2023)☆27Updated 11 months ago
- Embedding Recycling for Language models☆37Updated 2 years ago
- Plug-and-play Search Interfaces with Pyserini and Hugging Face☆32Updated 2 years ago
- Official Repository for "Hypencoder: Hypernetworks for Information Retrieval"☆31Updated 2 months ago
- ☆15Updated 2 months ago
- ☆13Updated 11 months ago
- Anchored Preference Optimization and Contrastive Revisions: Addressing Underspecification in Alignment☆60Updated last year
- A toolkit implementing advanced methods to transfer models and model knowledge across tokenizers.☆49Updated 4 months ago
- Set-Encoder: Permutation-Invariant Inter-Passage Attention for Listwise Passage Re-Ranking with Cross-Encoders☆18Updated 6 months ago
- MEXMA: Token-level objectives improve sentence representations☆42Updated 10 months ago
- ☆58Updated this week
- A massively multilingual modern encoder language model☆108Updated last month
- Fast search index for SPLADE sparse retrieval models implemented in Python using Numpy and Numba☆30Updated last month
- ☆22Updated 9 months ago
- SWIM-IR is a Synthetic Wikipedia-based Multilingual Information Retrieval training set with 28 million query-passage pairs spanning 33 la…☆49Updated 2 years ago
- Improving Text Embedding of Language Models Using Contrastive Fine-tuning☆65Updated last year
- Official implementation of "GPT or BERT: why not both?"☆62Updated 3 months ago
- https://footprints.baulab.info☆17Updated last year
- ☆55Updated 9 months ago
- Aioli: A unified optimization framework for language model data mixing☆28Updated 10 months ago
- PathPiece tokenizer☆13Updated last year
- [ACL 2025] Analyzing LLMs' Multilingual Knowledge Boundary Cognition Across Languages Through the Lens of Internal Representations☆14Updated last month
- Efficient encoder-decoder architecture for small language models (≤1B parameters) with cross-architecture knowledge distillation and visi…☆32Updated 9 months ago
- [EMNLP'23] Official Code for "FOCUS: Effective Embedding Initialization for Monolingual Specialization of Multilingual Models"☆34Updated 5 months ago
- Code for our paper Resources and Evaluations for Multi-Distribution Dense Information Retrieval☆15Updated last year
- 🕸 GlotCC Dataset and Pipline -- NeurIPS 2024☆20Updated 7 months ago
- QAmeleon introduces synthetic multilingual QA data using PaLM, a 540B large language model. This dataset was generated by prompt tuning P…☆35Updated 2 years ago