facebookresearch / distributed-faiss
A library for building and serving multi-node distributed faiss indices.
☆264Updated last year
Alternatives and similar repositories for distributed-faiss:
Users that are interested in distributed-faiss are comparing it to the libraries listed below
- Automatically create Faiss knn indices with the most optimal similarity search parameters.☆850Updated 11 months ago
- Some useful tips for faiss☆619Updated last year
- ⚡ A fast embedded library for approximate nearest neighbor search☆229Updated last year
- A large-scale information-rich web dataset, featuring millions of real clicked query-document labels☆323Updated 4 months ago
- Scalable training for dense retrieval models.☆292Updated last month
- Build Text Rerankers with Deep Language Models☆262Updated last year
- CUDA implementation of Hierarchical Navigable Small World Graph algorithm☆156Updated 4 years ago
- hnsw implemented by python☆66Updated 5 years ago
- Code used for sourcing and cleaning the BigScience ROOTS corpus☆309Updated 2 years ago
- Inquisitive Parrots for Search☆190Updated last year
- DSIR large-scale data selection framework for language model training☆246Updated last year
- Run Effective Large Batch Contrastive Learning Beyond GPU/TPU Memory Constraint☆386Updated last year
- ☆117Updated last year
- Running BERT without Padding☆471Updated 3 years ago
- ☆251Updated 9 months ago
- Code for "SemDeDup", a simple method for identifying and removing semantic duplicates from a dataset (data pairs which are semantically s…☆134Updated last year
- Implementation of a Transformer, but completely in Triton☆263Updated 3 years ago
- ☆246Updated last week
- ☆411Updated last year
- Code repository for the paper - "Matryoshka Representation Learning"☆487Updated last year
- Code repository for the paper - "AdANNS: A Framework for Adaptive Semantic Search"☆64Updated last year
- Serving multiple LoRA finetuned LLM as one☆1,054Updated 11 months ago
- Merlin Systems provides tools for combining recommendation models with other elements of production recommender systems (like feature sto…☆91Updated 10 months ago
- Codebase for RetroMAE and beyond.☆258Updated 10 months ago
- Search Engines with Autoregressive Language models☆284Updated 2 years ago
- Slicing a PyTorch Tensor Into Parallel Shards☆298Updated 3 years ago
- experiments with inference on llama☆104Updated 10 months ago
- Tevatron - Unified Document Retrieval Toolkit across Scale, Language, and Modality. Demo in SIGIR 2023, SIGIR 2025.☆584Updated this week
- Code used for the creation of OBELICS, an open, massive and curated collection of interleaved image-text web documents, containing 141M d…☆200Updated 7 months ago
- Official repository for DistFlashAttn: Distributed Memory-efficient Attention for Long-context LLMs Training☆209Updated 8 months ago