Preemo-Inc / text-generation-inference

Related projects ⓘ

Alternatives and complementary repositories for text-generation-inference

• hamelsmu / llama-inference
  experiments with inference on llama
  ☆105Updated 5 months ago
• AblateIt / finetune-study
  Comprehensive analysis of difference in performance of QLora, Lora, and Full Finetunes.
  ☆81Updated last year
• premAI-io / benchmarks
  🕹️ Performance Comparison of MLOps Engines, Frameworks, and Languages on Mainstream AI Models.
  ☆134Updated 3 months ago
• cognitivecomputations / laserRMT
  This is our own implementation of 'Layer Selective Rank Reduction'
  ☆232Updated 5 months ago
• cognitivecomputations / spectrum
  ☆94Updated last month
• neuralmagic / guidellm
  Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs
  ☆167Updated 2 weeks ago
• AnswerDotAI / fastdata
  ☆130Updated this week
• mixedbread-ai / batched
  ☆112Updated this week
• center-for-humans-and-machines / transformer-heads
  Toolkit for attaching, training, saving and loading of new heads for transformer models
  ☆246Updated 2 weeks ago
• taprosoft / llm_finetuning
  Convenient wrapper for fine-tuning and inference of Large Language Models (LLMs) with several quantization techniques (GTPQ, bitsandbytes…
  ☆145Updated last year
• cfahlgren1 / observers
  Track OpenAI compatible requests to a dataset
  ☆57Updated this week
• huggingface / llm-swarm
  Manage scalable open LLM inference endpoints in Slurm clusters
  ☆238Updated 4 months ago
• thomasgauthier / LoRD
  Low-Rank adapter extraction for fine-tuned transformers model
  ☆162Updated 6 months ago
• FastEval / FastEval
  Fast & more realistic evaluation of chat language models. Includes leaderboard.
  ☆183Updated 11 months ago
• huggingface / optimum-tpu
  Google TPU optimizations for transformers models
  ☆75Updated this week
• arcee-ai / DALM
  Domain Adapted Language Modeling Toolkit - E2E RAG
  ☆312Updated 2 weeks ago
• abacaj / train-with-fsdp
  ☆91Updated last year
• run-ai / runai-model-streamer
  ☆123Updated this week
• Locutusque / TPU-Alignment
  Fully fine-tune large models like Mistral, Llama-2-13B, or Qwen-14B completely for free
  ☆222Updated 3 weeks ago
• rmihaylov / mpttune
  Tune MPTs
  ☆84Updated last year
• TheBlokeAI / AIScripts
  Some simple scripts that I use day-to-day when working with LLMs and Huggingface Hub
  ☆155Updated last year
• huggingface / data-is-better-together
  Let's build better datasets, together!
  ☆206Updated this week
• michaelfeil / embed
  A stable, fast and easy-to-use inference library with a focus on a sync-to-async API
  ☆45Updated last month
• lightonai / pylate
  Late Interaction Models Training & Retrieval
  ☆166Updated this week
• jondurbin / bagel
  A bagel, with everything.
  ☆312Updated 7 months ago
• cognitivecomputations / kraken
  ☆64Updated 5 months ago
• teknium1 / LLM-Benchmark-Logs
  Just a bunch of benchmark logs for different LLMs
  ☆116Updated 3 months ago
• Mihaiii / llm_steer
  Steer LLM outputs towards a certain topic/subject and enhance response capabilities using activation engineering by adding steering vecto…
  ☆203Updated 6 months ago
• Gryphe / BlockMerge_Gradient
  Merge Transformers language models by use of gradient parameters.
  ☆201Updated 3 months ago
• neuralmagic / nm-vllm
  A high-throughput and memory-efficient inference and serving engine for LLMs
  ☆253Updated last month