run-ai / runai-model-streamer

Related projects ⓘ

Alternatives and complementary repositories for runai-model-streamer

• cognitivecomputations / kraken
  ☆64Updated 5 months ago
• neuralmagic / guidellm
  Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs
  ☆160Updated last week
• VITA-Group / Q-GaLore
  Q-GaLore: Quantized GaLore with INT4 Projection and Layer-Adaptive Low-Rank Gradients.
  ☆172Updated 3 months ago
• huggingface / optimum-tpu
  Google TPU optimizations for transformers models
  ☆74Updated this week
• cognitivecomputations / spectrum
  ☆92Updated last month
• teknium1 / ShareGPT-Builder
  ☆104Updated 7 months ago
• Mihaiii / backtrack_sampler
  An easy-to-understand framework for LLM samplers that rewind and revise generated tokens
  ☆105Updated 2 weeks ago
• samchaineau / llm_slerp_generation
  Repo hosting codes and materials related to speeding LLMs' inference using token merging.
  ☆29Updated 6 months ago
• cognitivecomputations / grokadamw
  ☆116Updated 2 months ago
• louisbrulenaudet / ragoon
  High level library for batched embeddings generation, blazingly-fast web-based RAG and quantized indexes processing ⚡
  ☆59Updated last week
• mani-kantap / llm-inference-solutions
  A collection of all available inference solutions for the LLMs
  ☆72Updated last month
• Not-Diamond / RoRF
  Routing on Random Forest (RoRF)
  ☆83Updated last month
• willccbb / mlx_parallm
  Fast parallel LLM inference for MLX
  ☆145Updated 4 months ago
• teknium1 / LLM-Benchmark-Logs
  Just a bunch of benchmark logs for different LLMs
  ☆114Updated 3 months ago
• Pleias / Quest-Best-Tokens
  An introduction to LLM Sampling
  ☆61Updated this week
• GreenBitAI / green-bit-llm
  A toolkit for fine-tuning, inferencing, and evaluating GreenBitAI's LLMs.
  ☆73Updated 3 weeks ago
• austinsilveria / tricksy
  Fast approximate inference on a single GPU with sparsity aware offloading
  ☆38Updated 10 months ago
• LLM360 / amber-data-prep
  Data preparation code for Amber 7B LLM
  ☆82Updated 6 months ago
• fw-ai / benchmark
  Benchmark suite for LLMs from Fireworks.ai
  ☆58Updated this week
• premAI-io / benchmarks
  🕹️ Performance Comparison of MLOps Engines, Frameworks, and Languages on Mainstream AI Models.
  ☆133Updated 3 months ago
• matthewrenze / jhu-concise-cot
  The Benefits of a Concise Chain of Thought on Problem Solving in Large Language Models
  ☆19Updated 9 months ago
• huggingface / llm-swarm
  Manage scalable open LLM inference endpoints in Slurm clusters
  ☆237Updated 4 months ago
• arcee-ai / EvolKit
  EvolKit is an innovative framework designed to automatically enhance the complexity of instructions used for fine-tuning Large Language M…
  ☆173Updated last week
• mzbac / mlx-moe
  Scripts to create your own moe models using mlx
  ☆86Updated 8 months ago
• argilla-io / argilla-cookbook
  Simple examples using Argilla tools to build AI
  ☆38Updated last week
• HazyResearch / lolcats
  Repo for "LoLCATs: On Low-Rank Linearizing of Large Language Models"
  ☆171Updated 3 weeks ago
• apple / ml-recurrent-drafter
  ☆96Updated last month
• arcee-ai / DAM
  ☆38Updated this week
• deshwalmahesh / PHUDGE
  Official repo for the paper PHUDGE: Phi-3 as Scalable Judge. Evaluate your LLMs with or without custom rubric, reference answer, absolute…
  ☆48Updated 4 months ago
• nyunAI / PruneGPT
  ☆52Updated 5 months ago