Alternatives and similar repositories for llm-profiling-toolkit

Users that are interested in llm-profiling-toolkit are comparing it to the libraries listed below

• cohere-ai / magikarp
  Code for the paper "Fishing for Magikarp"
  ☆162Updated 3 months ago
• MinishLab / tokenlearn
  Pre-train Static Word Embeddings
  ☆85Updated 2 months ago
• jxmorris12 / bm25_pt
  minimal pytorch implementation of bm25 (with sparse tensors)
  ☆104Updated last year
• EleutherAI / tokengrams
  Efficiently computing & storing token n-grams from large corpora
  ☆26Updated 10 months ago
• HazyResearch / cartridges
  Storing long contexts in tiny caches with self-study
  ☆124Updated this week
• jackbandy / bookcorpus-datasheet
  Documentation effort for the BookCorpus dataset
  ☆34Updated 4 years ago
• Pleias / Quest-Best-Tokens
  An introduction to LLM Sampling
  ☆79Updated 8 months ago
• taylorai / onnx_embedding_models
  utilities for loading and running text embeddings with onnx
  ☆44Updated last year
• haizelabs / thorn-in-haizestack
  Thorn in a HaizeStack test for evaluating long-context adversarial robustness.
  ☆26Updated last year
• bminixhofer / tokenkit
  A toolkit implementing advanced methods to transfer models and model knowledge across tokenizers.
  ☆41Updated last month
• luyug / magix
  Supercharge huggingface transformers with model parallelism.
  ☆77Updated 3 weeks ago
• EleutherAI / improved-t5
  Experiments for efforts to train a new and improved t5
  ☆76Updated last year
• Muhtasham / summarization-eval
  📝 Reference-Free automatic summarization evaluation with potential hallucination detection
  ☆101Updated last year
• davanstrien / haiku-dpo
  Using open source LLMs to build synthetic datasets for direct preference optimization
  ☆65Updated last year
• haizelabs / sphynx
  Sphynx Hallucination Induction
  ☆53Updated 6 months ago
• raphaelsty / LeNLP
  NLP with Rust for Python 🦀🐍
  ☆64Updated 3 months ago
• kpu / fasterText
  Library for fast text representation and classification.
  ☆31Updated last year
• castorini / hf-spacerini
  Plug-and-play Search Interfaces with Pyserini and Hugging Face
  ☆32Updated 2 years ago
• liujch1998 / infini-gram
  ☆61Updated last week
• AnswerDotAI / ModernBERT-Instruct-mini-cookbook
  ☆49Updated 6 months ago
• NathanGodey / headless-lm
  Training and evaluation code for the paper "Headless Language Models: Learning without Predicting with Contrastive Weight Tying" (https:/…
  ☆27Updated last year
• stephantul / unitoken
  Tokenization across languages. Useful as preprocessing for subword tokenization.
  ☆22Updated 2 years ago
• jxmorris12 / embzip
  lossily compress representation vectors using product quantization
  ☆58Updated 3 months ago
• Pleias / Pleias-RAG-Library
  Python library to use Pleias-RAG models
  ☆61Updated 3 months ago
• CarperAI / squeakily
  A library for squeakily cleaning and filtering language datasets.
  ☆47Updated 2 years ago
• Pleias / marginalia
  ☆67Updated last year
• google-research-datasets / seahorse
  Seahorse is a dataset for multilingual, multi-faceted summarization evaluation. It consists of 96K summaries with human ratings along 6 q…
  ☆88Updated last year
• google-deepmind / mishax
  ☆136Updated 4 months ago
• minosvasilias / simple_grpo
  Simple GRPO scripts and configurations.
  ☆59Updated 6 months ago
• mozilla-ai / lm-buddy
  Your buddy in the (L)LM space.
  ☆64Updated 10 months ago