Alternatives and similar repositories for institutional-books-1-pipeline

Users that are interested in institutional-books-1-pipeline are comparing it to the libraries listed below

• furiousteabag / vram-calculator
  Transformer GPU VRAM estimator
  ☆66Updated last year
• r-three / common-pile
  Code for collecting, processing, and preparing datasets for the Common Pile
  ☆234Updated last month
• cfahlgren1 / hf-data-explorer
  Chrome Extension for exploring Hugging Face datasets 🔎
  ☆48Updated last year
• Pleias / Pleias-RAG-Library
  Python library to use Pleias-RAG models
  ☆63Updated 5 months ago
• METR / eval-analysis-public
  Public repository containing METR's DVC pipeline for eval data analysis
  ☆120Updated 6 months ago
• mobiusml / aana_sdk
  Aana SDK is a powerful framework for building AI enabled multimodal applications.
  ☆52Updated last month
• epfl-dlab / GPTurk
  ☆29Updated 2 years ago
• babylm / baseline-pretraining
  Code for pre-training BabyLM baseline models.
  ☆16Updated 2 years ago
• simonw / llm-cluster
  LLM plugin for clustering embeddings
  ☆82Updated last year
• codelion / pts
  Pivotal Token Search
  ☆128Updated 3 months ago
• jxmorris12 / embzip
  lossily compress representation vectors using product quantization
  ☆59Updated 5 months ago
• eliahuhorwitz / Model-Atlas
  ☆30Updated 6 months ago
• scaleapi / SWE-bench_Pro-os
  SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks?
  ☆190Updated last week
• mungg / FABLES
  ☆57Updated last year
• MinishLab / tokenlearn
  Pre-train Static Word Embeddings
  ☆87Updated last month
• liujch1998 / infini-gram
  ☆72Updated 2 months ago
• KorAP / Tokenizer-Evaluation
  Benchmark scripts for comparing different tokenizers and sentence segmenters of German
  ☆12Updated 2 years ago
• EleutherAI / tokengrams
  Efficiently computing & storing token n-grams from large corpora
  ☆26Updated last year
• Pleias / OCRoscope
  Small python package to measure OCR quality and other related metrics.
  ☆25Updated last year
• Snowflake-Labs / arctic-embed
  ☆78Updated 10 months ago
• allenai / olmo-cookbook
  OLMost every training recipe you need to perform data interventions with the OLMo family of models.
  ☆50Updated last week
• JaderDias / download-from-common-crawl
  ☆23Updated last year
• mozilla-ai / lm-buddy
  Your buddy in the (L)LM space.
  ☆64Updated last year
• arXiv / arxiv-browse
  Flask app for article abstract and listing pages
  ☆172Updated this week
• lightonai / pylate-rs
  PyLate efficient inference engine
  ☆66Updated last month
• huggingface / huggingface-inference-toolkit
  Hugging Face Inference Toolkit used to serve transformers, sentence-transformers, and diffusers models.
  ☆87Updated last month
• ibm-granite / granite-embedding-models
  ☆43Updated last month
• simonw / llm-gguf
  Run models distributed as GGUF files using LLM
  ☆76Updated 10 months ago
• foundation-model-stack / bamba
  Train, tune, and infer Bamba model
  ☆134Updated 4 months ago
• DeployQL / LintDB
  Vector Database with support for late interaction and token level embeddings.
  ☆55Updated 3 months ago