IBM / text-generation-inference

Related projects: ⓘ

• fw-ai / benchmark
  Benchmark suite for LLMs from Fireworks.ai
  ☆51Updated this week
• huggingface / optimum-tpu
  Google TPU optimizations for transformers models
  ☆62Updated this week
• hamelsmu / llama-inference
  experiments with inference on llama
  ☆106Updated 3 months ago
• cognitivecomputations / spectrum
  ☆75Updated 3 weeks ago
• instructlab / sdg
  Python library for Synthetic Data Generation
  ☆13Updated last week
• bentoml / BentoVLLM
  Self-host LLMs with vLLM and BentoML
  ☆62Updated this week
• substratusai / vllm-docker
  ☆43Updated 3 weeks ago
• DeployQL / LintDB
  Vector Database with support for late interaction and token level embeddings.
  ☆51Updated last week
• ibm-granite / dolomite-engine
  ☆37Updated this week
• withmartian / routerbench
  The code for the paper ROUTERBENCH: A Benchmark for Multi-LLM Routing System
  ☆86Updated 3 months ago
• cognitivecomputations / kraken
  ☆64Updated 3 months ago
• IBM / data-prep-kit
  Open source project for data preparation of LLM application builders
  ☆124Updated this week
• IBM / ensemble-instruct
  codebase release for EMNLP2023 paper publication
  ☆19Updated 6 months ago
• lamm-mit / Cephalo-Phi-3-Vision-MoE
  ☆11Updated 3 months ago
• mani-kantap / llm-inference-solutions
  A collection of all available inference solutions for the LLMs
  ☆65Updated 2 weeks ago
• golololologol / LLM-Distillery
  A pipeline for LLM knowledge distillation
  ☆68Updated last month
• premAI-io / benchmarks
  🕹️ Performance Comparison of MLOps Engines, Frameworks, and Languages on Mainstream AI Models.
  ☆129Updated last month
• neuralmagic / guidellm
  Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs
  ☆128Updated this week
• LLM360 / amber-data-prep
  Data preparation code for Amber 7B LLM
  ☆76Updated 4 months ago
• ai-hero / llm-research-fine-tuning
  ☆15Updated 3 months ago
• patronus-ai / Lynx-hallucination-detection
  ☆24Updated 2 months ago
• asprenger / ray_vllm_inference
  A simple service that integrates vLLM with Ray Serve for fast and scalable LLM serving.
  ☆45Updated 5 months ago
• LLM360 / Analysis360
  Evaluation and analysis code for LLM360
  ☆75Updated 3 months ago
• philschmid / text-generation-inference-tests
  ☆20Updated 8 months ago
• EmbeddedLLM / vllm-rocm
  vLLM: A high-throughput and memory-efficient inference and serving engine for LLMs
  ☆87Updated this week
• samchaineau / llm_slerp_generation
  Repo hosting codes and materials related to speeding LLMs' inference using token merging.
  ☆28Updated 4 months ago
• IBM / unitxt
  🦄 Unitxt: a python library for getting data fired up and set for training and evaluation
  ☆151Updated this week
• Preemo-Inc / text-generation-inference
  ☆201Updated 7 months ago
• teknium1 / LLM-Benchmark-Logs
  Just a bunch of benchmark logs for different LLMs
  ☆112Updated last month
• mixedbread-ai / batched
  ☆58Updated 3 weeks ago