LAGoM-NLP / transtokenizerLinks
☆50Updated 6 months ago
Alternatives and similar repositories for transtokenizer
Users that are interested in transtokenizer are comparing it to the libraries listed below
Sorting:
- Trully flash implementation of DeBERTa disentangled attention mechanism.☆62Updated 2 months ago
- Simple replication of [ColBERT-v1](https://arxiv.org/abs/2004.12832).☆80Updated last year
- A toolkit implementing advanced methods to transfer models and model knowledge across tokenizers.☆40Updated 3 weeks ago
- Code for Zero-Shot Tokenizer Transfer☆133Updated 6 months ago
- Pre-train Static Word Embeddings☆85Updated 2 months ago
- ☆56Updated 2 months ago
- ☆79Updated 2 months ago
- Notebooks for training universal 0-shot classifiers on many different tasks☆133Updated 7 months ago
- Improving Text Embedding of Language Models Using Contrastive Fine-tuning☆64Updated last year
- SWIM-IR is a Synthetic Wikipedia-based Multilingual Information Retrieval training set with 28 million query-passage pairs spanning 33 la…☆49Updated last year
- MEXMA: Token-level objectives improve sentence representations☆41Updated 6 months ago
- Anchored Preference Optimization and Contrastive Revisions: Addressing Underspecification in Alignment☆60Updated 11 months ago
- ☆57Updated 10 months ago
- Using open source LLMs to build synthetic datasets for direct preference optimization☆65Updated last year
- QAmeleon introduces synthetic multilingual QA data using PaLM, a 540B large language model. This dataset was generated by prompt tuning P…☆34Updated last year
- BPE modification that implements removing of the intermediate tokens during tokenizer training.☆24Updated 8 months ago
- The first dense retrieval model that can be prompted like an LM☆81Updated 2 months ago
- Official implementation of "GPT or BERT: why not both?"☆56Updated last week
- Training and evaluation code for the paper "Headless Language Models: Learning without Predicting with Contrastive Weight Tying" (https:/…☆27Updated last year
- Code and data releases for the paper -- DelTA: An Online Document-Level Translation Agent Based on Multi-Level Memory☆45Updated 5 months ago
- Hugging Face Inference Toolkit used to serve transformers, sentence-transformers, and diffusers models.☆83Updated last week
- Plug-and-play Search Interfaces with Pyserini and Hugging Face☆32Updated last year
- Minimum Bayes Risk Decoding for Hugging Face Transformers☆58Updated last year
- Backtracing: Retrieving the Cause of the Query, EACL 2024 Long Paper, Findings.☆89Updated last year
- Official Repository for "Hypencoder: Hypernetworks for Information Retrieval"☆27Updated 4 months ago
- Maya: An Instruction Finetuned Multilingual Multimodal Model using Aya☆117Updated 2 weeks ago
- Optimus is a flexible and scalable framework built to train language models efficiently across diverse hardware configurations, including…☆66Updated last month
- ☆49Updated 5 months ago
- Starbucks: Improved Training for 2D Matryoshka Embeddings☆21Updated last month
- 💬 Language Identification with Support for More Than 2000 Labels -- EMNLP 2023☆147Updated last month