Alternatives and similar repositories for llm-profiling-toolkit

Users that are interested in llm-profiling-toolkit are comparing it to the libraries listed below

• jxmorris12 / embzip
  lossily compress representation vectors using product quantization
  ☆59Updated 3 weeks ago
• EleutherAI / tokengrams
  Efficiently computing & storing token n-grams from large corpora
  ☆26Updated last year
• cohere-ai / magikarp
  Code for the paper "Fishing for Magikarp"
  ☆174Updated 6 months ago
• CarperAI / squeakily
  A library for squeakily cleaning and filtering language datasets.
  ☆48Updated 2 years ago
• bminixhofer / tokenkit
  A toolkit implementing advanced methods to transfer models and model knowledge across tokenizers.
  ☆49Updated 4 months ago
• jackbandy / bookcorpus-datasheet
  Documentation effort for the BookCorpus dataset
  ☆34Updated 4 years ago
• Mihaiii / backtrack_sampler
  An easy-to-understand framework for LLM samplers that rewind and revise generated tokens
  ☆145Updated 9 months ago
• Aleph-Alpha-Research / scaling
  Scaling is a distributed training library and installable dependency designed to scale up neural networks, with a dedicated module for tr…
  ☆64Updated last month
• CarperAI / decontamination
  This repository contains code for cleaning your training data of benchmark data to help combat data snooping.
  ☆27Updated 2 years ago
• haizelabs / sphynx
  Sphynx Hallucination Induction
  ☆53Updated 9 months ago
• MinishLab / tokenlearn
  Pre-train Static Word Embeddings
  ☆90Updated 2 months ago
• Muhtasham / summarization-eval
  📝 Reference-Free automatic summarization evaluation with potential hallucination detection
  ☆102Updated last year
• JD-P / RetroInstruct
  Synthetic data derived by templating, few shot prompting, transformations on public domain corpora, and monte carlo tree search.
  ☆32Updated last month
• mkuchnik / relm
  ReLM is a Regular Expression engine for Language Models
  ☆107Updated 2 years ago
• catid / lllm
  Latent Large Language Models
  ☆19Updated last year
• AnswerDotAI / ModernBERT-Instruct-mini-cookbook
  ☆51Updated 9 months ago
• teknium1 / transformers-gptq-quant
  ☆45Updated 2 years ago
• raphaelsty / LeNLP
  NLP with Rust for Python 🦀🐍
  ☆66Updated 6 months ago
• EleutherAI / improved-t5
  Experiments for efforts to train a new and improved t5
  ☆75Updated last year
• jxmorris12 / bm25_pt
  minimal pytorch implementation of bm25 (with sparse tensors)
  ☆104Updated 3 weeks ago
• taylorai / onnx_embedding_models
  utilities for loading and running text embeddings with onnx
  ☆44Updated 3 months ago
• haizelabs / thorn-in-haizestack
  Thorn in a HaizeStack test for evaluating long-context adversarial robustness.
  ☆26Updated last year
• Aleph-Alpha-Research / trigrams
  ☆58Updated this week
• RAIVNLab / AdANNS
  Code repository for the paper - "AdANNS: A Framework for Adaptive Semantic Search"
  ☆65Updated 2 years ago
• illuin-tech / contextual-embeddings
  Model implementation for the contextual embeddings project
  ☆36Updated 5 months ago
• joshuacnf / Ctrl-G
  ☆104Updated 10 months ago
• sileod / tasksource
  Datasets collection and preprocessings framework for NLP extreme multitask learning
  ☆188Updated 4 months ago
• geronimi73 / phi2-finetune
  ☆86Updated last year
• minosvasilias / simple_grpo
  Simple GRPO scripts and configurations.
  ☆59Updated 9 months ago
• ConsequentAI / fneval
  Functional Benchmarks and the Reasoning Gap
  ☆89Updated last year