intel / xFasterTransformer
☆348Updated 2 weeks ago
Related projects: ⓘ
- RTP-LLM: Alibaba's high-performance LLM inference engine for diverse applications.☆512Updated last week
- GLake: optimizing GPU memory management and IO transmission.☆351Updated last month
- DashInfer is a native LLM inference engine aiming to deliver industry-leading performance atop various hardware architectures, including …☆130Updated 3 weeks ago
- LLM Inference benchmark☆331Updated last month
- ☆251Updated last week
- Disaggregated serving system for Large Language Models (LLMs).☆278Updated last month
- Low-bit LLM inference on CPU with lookup table☆440Updated this week
- ☆123Updated 3 months ago
- AI Accelerator Benchmark focuses on evaluating AI Accelerators from a practical production perspective, including the ease of use and ver…☆188Updated 3 weeks ago
- FlagScale is a large model toolkit based on open-sourced projects.☆129Updated last week
- FlashInfer: Kernel Library for LLM Serving☆1,143Updated last week
- Analyze the inference of Large Language Models (LLMs). Analyze aspects like computation, storage, transmission, and hardware roofline mod…☆276Updated last week
- optimized BERT transformer inference on NVIDIA GPU. https://arxiv.org/abs/2210.03052☆451Updated 6 months ago
- ☆140Updated 4 months ago
- FlagGems is an operator library for large language models implemented in Triton Language.☆246Updated last week
- A throughput-oriented high-performance serving framework for LLMs☆470Updated this week
- Easy Parallel Library (EPL) is a general and efficient deep learning framework for distributed model training.☆260Updated last year
- Dynamic Memory Management for Serving LLMs without PagedAttention☆186Updated last month
- QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Serving☆399Updated 2 weeks ago
- Inferflow is an efficient and highly configurable inference engine for large language models (LLMs).☆232Updated 6 months ago
- The Triton TensorRT-LLM Backend☆654Updated this week
- FP16xINT4 LLM inference kernel that can achieve near-ideal ~4x speedups up to medium batchsizes of 16-32 tokens.☆562Updated 2 weeks ago
- A PyTorch Native LLM Training Framework☆581Updated 3 weeks ago
- Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.☆1,045Updated last month
- This is the official PyTorch implementation of "LLMC: Benchmarking Large Language Model Quantization with a Versatile Compression Toolkit…☆227Updated this week
- llm-export can export llm model to onnx.☆193Updated this week
- ☆133Updated 2 months ago
- A low-latency & high-throughput serving engine for LLMs☆174Updated last week
- Transformer related optimization, including BERT, GPT☆58Updated last year
- A high-performance inference system for large language models, designed for production environments.☆370Updated last week