ray-project / ray-llmLinks
RayLLM - LLMs on Ray (Archived). Read README for more info.
☆1,260Updated 2 months ago
Alternatives and similar repositories for ray-llm
Users that are interested in ray-llm are comparing it to the libraries listed below
Sorting:
- S-LoRA: Serving Thousands of Concurrent LoRA Adapters☆1,826Updated last year
- MII makes low-latency and high-throughput inference possible, powered by DeepSpeed.☆2,009Updated 2 months ago
- Serving multiple LoRA finetuned LLM as one☆1,060Updated last year
- LLMPerf is a library for validating and benchmarking LLMs☆921Updated 5 months ago
- ☆459Updated last year
- Python bindings for the Transformer models implemented in C/C++ using GGML library.☆1,864Updated last year
- Customizable implementation of the self-instruct paper.☆1,042Updated last year
- [ICML 2024] Break the Sequential Dependency of LLM Inference Using Lookahead Decoding☆1,249Updated 2 months ago
- Scale LLM Engine public repository☆799Updated 2 weeks ago
- AutoAWQ implements the AWQ algorithm for 4-bit quantization with a 2x speedup during inference. Documentation:☆2,174Updated 3 weeks ago
- Chat language model that can use tools and interpret the results☆1,555Updated this week
- Multi-LoRA inference server that scales to 1000s of fine-tuned LLMs☆2,985Updated last week
- Extend existing LLMs way beyond the original training length with constant memory usage, without retraining☆697Updated last year
- A tiny library for coding with large language models.☆1,232Updated 10 months ago
- Examples on how to use LangChain and Ray☆230Updated last year
- A high-performance inference system for large language models, designed for production environments.☆441Updated this week
- A more memory-efficient rewrite of the HF transformers implementation of Llama for use with quantized weights.☆2,874Updated last year
- Official repository for LongChat and LongEval☆518Updated last year
- batched loras☆342Updated last year
- CodeTF: One-stop Transformer Library for State-of-the-art Code LLM☆1,476Updated last month
- ☆971Updated 3 months ago
- [ACL 2023] One Embedder, Any Task: Instruction-Finetuned Text Embeddings☆1,954Updated 4 months ago
- This repository contains code and tooling for the Abacus.AI LLM Context Expansion project. Also included are evaluation scripts and bench…☆586Updated last year
- ☆451Updated last year
- Medusa: Simple Framework for Accelerating LLM Generation with Multiple Decoding Heads☆2,530Updated 11 months ago
- YaRN: Efficient Context Window Extension of Large Language Models☆1,489Updated last year
- ⚡ Build your chatbot within minutes on your favorite device; offer SOTA compression techniques for LLMs; run LLMs efficiently on Intel Pl…☆2,170Updated 7 months ago
- A tool for evaluating LLMs☆419Updated last year
- Code for the ICLR 2023 paper "GPTQ: Accurate Post-training Quantization of Generative Pretrained Transformers".☆2,119Updated last year
- Efficient Retrieval Augmentation and Generation Framework☆1,558Updated 4 months ago