Alternatives and similar repositories for happy_vllm:

Users that are interested in happy_vllm are comparing it to the libraries listed below

• IlyasMoutawwakil / py-txi
  A Python wrapper around HuggingFace's TGI (text-generation-inference) and TEI (text-embedding-inference) servers.
  ☆34Updated last month
• Zyphra / transformers_zamba2
  ☆42Updated this week
• HITsz-TMG / KaLM-Embedding
  Code for KaLM-Embedding models
  ☆64Updated last week
• cxcscmu / RAGViz
  Official repository for RAGViz: Diagnose and Visualize Retrieval-Augmented Generation [EMNLP 2024]
  ☆76Updated this week
• milvus-io / milvus-model
  A library integrating embedding and reranker models from OpenAI, SentenceTransformers etc for semantic search in vector database.
  ☆29Updated this week
• aniketmaurya / fastserve-ai
  Machine Learning Serving focused on GenAI with simplicity as the top priority.
  ☆58Updated 2 weeks ago
• LLM360 / crystalcoder-data-prep
  Data preparation code for CrystalCoder 7B LLM
  ☆44Updated 8 months ago
• phunterlau / paper_without_code
  LLM reads a paper and produce a working prototype
  ☆47Updated 3 weeks ago
• AlexBodner / How_Much_VRAM
  ☆96Updated 4 months ago
• IlyasMoutawwakil / llm-perf-backend
  The backend behind the LLM-Perf Leaderboard
  ☆10Updated 8 months ago
• substratusai / vllm-docker
  ☆50Updated 2 weeks ago
• etalab-ia / albert-models
  Deployment a light and full OpenAI API for production with vLLM to support /v1/embeddings with all embeddings models.
  ☆39Updated 6 months ago
• codefuse-ai / CodeFuse-CGE
  ☆17Updated 3 weeks ago
• kyegomez / Infini-attention
  Implementation of the paper: "Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention" from Google in pyTO…
  ☆53Updated last week
• tensorchord / inference-benchmark
  Benchmark for machine learning model online serving (LLM, embedding, Stable-Diffusion, Whisper)
  ☆27Updated last year
• nyunAI / PruneGPT
  ☆52Updated 7 months ago
• LLM-inference-router / vllm-router
  vLLM Router
  ☆17Updated 10 months ago
• kyegomez / FastFF
  Zeta implementation of a reusable and plug in and play feedforward from the paper "Exponentially Faster Language Modeling"
  ☆15Updated 2 months ago
• read-agent / read-agent.github.io
  ☆51Updated 6 months ago
• huggingface / huggingface-inference-toolkit
  Hugging Face Inference Toolkit used to serve transformers, sentence-transformers, and diffusers models.
  ☆61Updated last week
• foundation-model-stack / bamba
  Train, tune, and infer Bamba model
  ☆77Updated last week
• golololologol / LLM-Distillery
  A pipeline for LLM knowledge distillation
  ☆83Updated 5 months ago
• leloykun / modded-nanogpt
  NanoGPT (124M) quality in 2.67B tokens
  ☆26Updated this week
• appl-team / reppl
  Using APPL to reimplement popular algorithms for Large Language Models (LLMs) and prompts
  ☆43Updated last week
• redis-developer / LLM-Document-Chat
  Using LlamaIndex, Redis, and OpenAI to chat with PDF documents. Supplementary material for blog post on Microsoft Developer Blog
  ☆110Updated last year
• mmhamdy / open-language-models
  A list of language models with permissive licenses such as MIT or Apache 2.0
  ☆24Updated 2 months ago
• opendatahub-io / vllm-tgis-adapter
  vLLM adapter for a TGIS-compatible gRPC server.
  ☆16Updated this week
• asprenger / ray_vllm_inference
  A simple service that integrates vLLM with Ray Serve for fast and scalable LLM serving.
  ☆60Updated 9 months ago
• fw-ai / benchmark
  Benchmark suite for LLMs from Fireworks.ai
  ☆64Updated last month