facebookresearch / distributed-faissLinks
A library for building and serving multi-node distributed faiss indices.
☆268Updated last year
Alternatives and similar repositories for distributed-faiss
Users that are interested in distributed-faiss are comparing it to the libraries listed below
Sorting:
- Some useful tips for faiss☆620Updated last year
- Automatically create Faiss knn indices with the most optimal similarity search parameters.☆868Updated last year
- ⚡ A fast embedded library for approximate nearest neighbor search☆233Updated 2 years ago
- hnsw implemented by python☆69Updated 6 years ago
- A large-scale information-rich web dataset, featuring millions of real clicked query-document labels☆340Updated 7 months ago
- Scalable training for dense retrieval models.☆299Updated 2 months ago
- ☆412Updated last year
- Framework for evaluating ANNS algorithms on billion scale datasets.☆391Updated 3 months ago
- An efficient implementation of the popular sequence models for text generation, summarization, and translation tasks. https://arxiv.org/p…☆433Updated 2 years ago
- Code used for sourcing and cleaning the BigScience ROOTS corpus☆313Updated 2 years ago
- Code repository for the paper - "Matryoshka Representation Learning"☆535Updated last year
- A minimal PyTorch Lightning OpenAI GPT w DeepSpeed Training!☆112Updated 2 years ago
- A large-scale multilingual dataset for Information Retrieval. Thorough human-annotations across 18 diverse languages.☆193Updated last year
- Scalable PaLM implementation of PyTorch☆190Updated 2 years ago
- Code repository for the paper - "AdANNS: A Framework for Adaptive Semantic Search"☆65Updated last year
- CUDA implementation of Hierarchical Navigable Small World Graph algorithm☆162Updated 4 years ago
- Open Instruction Generalist is an assistant trained on massive synthetic instructions to perform many millions of tasks☆208Updated last year
- Binary Passage Retriever (BPR) - an efficient passage retriever for open-domain question answering☆172Updated 4 years ago
- Search Engines with Autoregressive Language models☆291Updated 2 years ago
- Implementation of a Transformer, but completely in Triton☆273Updated 3 years ago
- Run Effective Large Batch Contrastive Learning Beyond GPU/TPU Memory Constraint☆401Updated last year
- Code for "SemDeDup", a simple method for identifying and removing semantic duplicates from a dataset (data pairs which are semantically s…☆139Updated last year
- A scalable & efficient active learning/data selection system for everyone.☆215Updated last year
- The pipeline for the OSCAR corpus☆171Updated last year
- ☆120Updated last year
- Inquisitive Parrots for Search☆194Updated 2 months ago
- experiments with inference on llama☆104Updated last year
- docTTTTTquery document expansion model☆368Updated 2 years ago
- XtremeDistil framework for distilling/compressing massive multilingual neural network models to tiny and efficient models for AI at scale☆155Updated last year
- DSIR large-scale data selection framework for language model training☆258Updated last year