facebookresearch / distributed-faissLinks
A library for building and serving multi-node distributed faiss indices.
☆269Updated last year
Alternatives and similar repositories for distributed-faiss
Users that are interested in distributed-faiss are comparing it to the libraries listed below
Sorting:
- Automatically create Faiss knn indices with the most optimal similarity search parameters.☆872Updated last year
- ⚡ A fast embedded library for approximate nearest neighbor search☆234Updated 2 years ago
- hnsw implemented by python☆70Updated 6 years ago
- A large-scale information-rich web dataset, featuring millions of real clicked query-document labels☆343Updated 10 months ago
- Scalable training for dense retrieval models.☆297Updated 4 months ago
- ☆412Updated last year
- Inquisitive Parrots for Search☆198Updated 4 months ago
- CUDA implementation of Hierarchical Navigable Small World Graph algorithm☆167Updated 4 years ago
- Code for "SemDeDup", a simple method for identifying and removing semantic duplicates from a dataset (data pairs which are semantically s…☆144Updated 2 years ago
- Code used for sourcing and cleaning the BigScience ROOTS corpus☆316Updated 2 years ago
- Code repository for the paper - "Matryoshka Representation Learning"☆567Updated last year
- Efficient, check-pointed data loading for deep learning with massive data sets.☆209Updated 2 years ago
- A scalable & efficient active learning/data selection system for everyone.☆217Updated last year
- A minimal PyTorch Lightning OpenAI GPT w DeepSpeed Training!☆113Updated 2 years ago
- A large-scale multilingual dataset for Information Retrieval. Thorough human-annotations across 18 diverse languages.☆196Updated last year
- Code repository for the paper - "AdANNS: A Framework for Adaptive Semantic Search"☆65Updated 2 years ago
- The pipeline for the OSCAR corpus☆173Updated last year
- Search Engines with Autoregressive Language models☆292Updated 2 years ago
- XtremeDistil framework for distilling/compressing massive multilingual neural network models to tiny and efficient models for AI at scale☆156Updated last year
- Implementation of a Transformer, but completely in Triton☆275Updated 3 years ago
- Binary Passage Retriever (BPR) - an efficient passage retriever for open-domain question answering☆172Updated 4 years ago
- Build Text Rerankers with Deep Language Models☆263Updated last year
- docTTTTTquery document expansion model☆371Updated 2 years ago
- Easily convert common crawl to a dataset of caption and document. Image/text Audio/text Video/text, ...☆319Updated last year
- The Triton backend for the PyTorch TorchScript models.☆160Updated last week
- Official repo to On the Generalization Ability of Retrieval-Enhanced Transformers☆44Updated last year
- 🛠️ Tools for Transformers compression using PyTorch Lightning ⚡☆85Updated 2 weeks ago
- Run Effective Large Batch Contrastive Learning Beyond GPU/TPU Memory Constraint☆413Updated last year
- Pure python implementation of product quantization for nearest neighbor search☆351Updated 4 months ago
- Official code for "Binary embedding based retrieval at Tencent"☆43Updated last year