facebookresearch / distributed-faissLinks
A library for building and serving multi-node distributed faiss indices.
☆269Updated last year
Alternatives and similar repositories for distributed-faiss
Users that are interested in distributed-faiss are comparing it to the libraries listed below
Sorting:
- Automatically create Faiss knn indices with the most optimal similarity search parameters.☆871Updated last year
- Some useful tips for faiss☆621Updated this week
- ⚡ A fast embedded library for approximate nearest neighbor search☆233Updated 2 years ago
- hnsw implemented by python☆69Updated 6 years ago
- A large-scale information-rich web dataset, featuring millions of real clicked query-document labels☆341Updated 8 months ago
- A minimal PyTorch Lightning OpenAI GPT w DeepSpeed Training!☆113Updated 2 years ago
- Scalable training for dense retrieval models.☆299Updated 2 months ago
- Framework for evaluating ANNS algorithms on billion scale datasets.☆393Updated last week
- Pure python implementation of product quantization for nearest neighbor search☆350Updated 2 months ago
- Code repository for the paper - "Matryoshka Representation Learning"☆546Updated last year
- Code used for sourcing and cleaning the BigScience ROOTS corpus☆313Updated 2 years ago
- A large-scale multilingual dataset for Information Retrieval. Thorough human-annotations across 18 diverse languages.☆194Updated last year
- The pipeline for the OSCAR corpus☆171Updated last year
- A scalable & efficient active learning/data selection system for everyone.☆215Updated last year
- A robust web archive analytics toolkit☆116Updated 5 months ago
- Binary Passage Retriever (BPR) - an efficient passage retriever for open-domain question answering☆172Updated 4 years ago
- Build Text Rerankers with Deep Language Models☆262Updated last year
- Code for "SemDeDup", a simple method for identifying and removing semantic duplicates from a dataset (data pairs which are semantically s…☆139Updated last year
- Search Engines with Autoregressive Language models☆291Updated 2 years ago
- Visualize hnsw, faiss and other anns index☆455Updated 2 years ago
- Provides a common interface to many IR ranking datasets.☆367Updated 2 months ago
- An efficient implementation of the popular sequence models for text generation, summarization, and translation tasks. https://arxiv.org/p…☆433Updated 3 years ago
- ☆412Updated last year
- Scalable PaLM implementation of PyTorch☆190Updated 2 years ago
- docTTTTTquery document expansion model☆368Updated 2 years ago
- This project studies the performance and robustness of language models and task-adaptation methods.☆151Updated last year
- Efficient, check-pointed data loading for deep learning with massive data sets.☆209Updated 2 years ago
- Inquisitive Parrots for Search☆196Updated 3 months ago
- Task-based datasets, preprocessing, and evaluation for sequence models.☆586Updated this week
- A novel embedding training algorithm leveraging ANN search and achieved SOTA retrieval on Trec DL 2019 and OpenQA benchmarks☆376Updated 2 years ago