anarchy-ai / llm-speed-benchmark

Related projects ⓘ

Alternatives and complementary repositories for llm-speed-benchmark

• unifyai / aibench-llm-endpoints
  Runner in charge of collecting metrics from LLM inference endpoints for the Unify Hub
  ☆17Updated 9 months ago
• mify-io / mify-llm-editor
  LLM code editor for backend services
  ☆11Updated last month
• IlyasMoutawwakil / llm-perf-backend
  The backend behind the LLM-Perf Leaderboard
  ☆10Updated 6 months ago
• teknium1 / LLM-Logbook
  Public reports detailing responses to sets of prompts by Large Language Models.
  ☆26Updated last year
• run-ai / vscode-genv
  GPU Environment Management for Visual Studio Code
  ☆35Updated last year
• fw-ai / cookbook
  Collection of recipes aiding Gen AI model development
  ☆88Updated last week
• ajtejankar / mixtral-vis-moe
  Visualize expert firing frequencies across sentences in the Mixtral MoE model
  ☆17Updated 10 months ago
• fw-ai / benchmark
  Benchmark suite for LLMs from Fireworks.ai
  ☆58Updated 2 weeks ago
• simonw / llm-openrouter
  LLM plugin for models hosted by OpenRouter
  ☆68Updated 6 months ago
• discus-labs / discus
  A data-centric AI package for ML/AI. Get the best high-quality data for the best results. Discord: https://discord.gg/t6ADqBKrdZ
  ☆63Updated last year
• premAI-io / prem-operator
  📡 Deploy AI models and apps to Kubernetes without developing a hernia
  ☆31Updated 5 months ago
• Alignment-Lab-AI / datagen
  a pipeline for using api calls to agnostically convert unstructured data into structured training data
  ☆28Updated 2 months ago
• Preemo-Inc / text-generation-inference
  ☆200Updated 9 months ago
• furiousteabag / vram-calculator
  Transformer GPU VRAM estimator
  ☆40Updated 7 months ago
• the-crypt-keeper / tiny_starcoder
  Python examples using the bigcode/tiny_starcoder_py 159M model to generate code
  ☆44Updated last year
• anyscale / llm-router
  Tutorial for building LLM router
  ☆163Updated 4 months ago
• tensorchord / ai-infra-landscape
  This is a landscape of the infrastructure that powers the generative AI ecosystem
  ☆130Updated last month
• titanml / takeoff-community
  TitanML Takeoff Server is an optimization, compression and deployment platform that makes state of the art machine learning models access…
  ☆114Updated 9 months ago
• TheBlokeAI / AIScripts
  Some simple scripts that I use day-to-day when working with LLMs and Huggingface Hub
  ☆155Updated last year
• inferless / triton-co-pilot
  Generate Glue Code in seconds to simplify your Nvidia Triton Inference Server Deployments
  ☆15Updated 4 months ago
• mkuchnik / relm
  ReLM is a Regular Expression engine for Language Models
  ☆104Updated last year
• teknium1 / LLM-Benchmark-Logs
  Just a bunch of benchmark logs for different LLMs
  ☆115Updated 3 months ago
• asprenger / ray_vllm_inference
  A simple service that integrates vLLM with Ray Serve for fast and scalable LLM serving.
  ☆54Updated 7 months ago
• okuvshynov / cubestat
  Horizon chart for CPU/GPU/Neural Engine utilization monitoring on Apple M1/M2 and nVidia GPUs on Linux
  ☆24Updated last month
• huggingface / optimum-tpu
  Google TPU optimizations for transformers models
  ☆75Updated this week
• kyegomez / Exa
  Unleash the full potential of exascale LLMs on consumer-class GPUs, proven by extensive benchmarks, with no long-term adjustments and min…
  ☆23Updated last week
• emrgnt-cmplxty / zero-shot-replication
  ☆72Updated last year
• toshsan / embedding-server
  Drop in replacement for OpenAI's embedding API. Self Hosted.
  ☆51Updated last year
• bentoml / BentoVLLM
  Self-host LLMs with vLLM and BentoML
  ☆74Updated last week
• neoxelox / dspy-inspector
  DSPy program/pipeline inspector widget for Jupyter/VSCode Notebooks.
  ☆28Updated 9 months ago