facebookresearch / distributed-faiss
A library for building and serving multi-node distributed faiss indices.
☆262Updated last year
Alternatives and similar repositories for distributed-faiss:
Users that are interested in distributed-faiss are comparing it to the libraries listed below
- Automatically create Faiss knn indices with the most optimal similarity search parameters.☆834Updated 9 months ago
- ⚡ A fast embedded library for approximate nearest neighbor search☆226Updated last year
- Framework for evaluating ANNS algorithms on billion scale datasets.☆364Updated this week
- Some useful tips for faiss☆616Updated last year
- Code used for sourcing and cleaning the BigScience ROOTS corpus☆307Updated last year
- hnsw implemented by python☆64Updated 5 years ago
- CUDA implementation of Hierarchical Navigable Small World Graph algorithm☆154Updated 3 years ago
- This project studies the performance and robustness of language models and task-adaptation methods.☆144Updated 9 months ago
- Scalable training for dense retrieval models.☆275Updated last year
- A large-scale information-rich web dataset, featuring millions of real clicked query-document labels☆313Updated 2 months ago
- Fast Inference Solutions for BLOOM☆563Updated 4 months ago
- DSIR large-scale data selection framework for language model training☆241Updated 10 months ago
- Code repository for the paper - "AdANNS: A Framework for Adaptive Semantic Search"☆63Updated last year
- ☆410Updated last year
- Code repository for supporting the paper "Atlas Few-shot Learning with Retrieval Augmented Language Models",(https//arxiv.org/abs/2208.03…☆526Updated last year
- Build Text Rerankers with Deep Language Models☆258Updated last year
- Tevatron - A flexible toolkit for neural retrieval research and development.☆561Updated this week
- Open Instruction Generalist is an assistant trained on massive synthetic instructions to perform many millions of tasks☆208Updated last year
- ☆226Updated this week
- Code for ECCV2018 paper: Revisiting the Inverted Indices for Billion-Scale Approximate Nearest Neighbors☆203Updated 4 years ago
- Visualize hnsw, faiss and other anns index☆421Updated last year
- Codebase for RetroMAE and beyond.☆249Updated 8 months ago
- A large-scale multilingual dataset for Information Retrieval. Thorough human-annotations across 18 diverse languages.☆173Updated 6 months ago
- ☆69Updated last month
- Easily convert common crawl to a dataset of caption and document. Image/text Audio/text Video/text, ...☆315Updated last year
- Pytorch implementation of DoReMi, a method for optimizing the data mixture weights in language modeling datasets☆312Updated last year
- Run Effective Large Batch Contrastive Learning Beyond GPU/TPU Memory Constraint☆374Updated 10 months ago
- Code for "SemDeDup", a simple method for identifying and removing semantic duplicates from a dataset (data pairs which are semantically s…☆125Updated last year
- Inquisitive Parrots for Search☆186Updated 11 months ago
- Code repository for the paper - "Matryoshka Representation Learning"☆459Updated last year